Revolutionising virtual reality: enabling human-like NPC interactions through Microsoft Azure OpenAI-powered large language models

This project is planned to produce an AI-powered NPC dialogue system that significantly enhances user engagement and learning outcomes in Virtual Reality (VR) Delivering Bad News Module Application. This research and development effort aims to improve user-NPC interactions in VR environments by adding Large Language Models (LLMs). The Delivering Bad News module will be the focus, with the goal of allowing NPCs to engage in realistic and contextually relevant dialogue. This VR is to help user to practice how to handle challenging and difficult situations when communicating with difficult employees. Many VR systems now display NPC interactions that are unrealistic or changeable/static, making them less effective at reproducing real-world scenarios. These limits are addressed with advanced LLMs such as Azure OpenAI's Generative Pre-Trained Transformer (GPT), which provides a more engaging user experience. The project employs an Agile methodology, with an iterative development process that incorporates continuous feedback and testing to improve the NPC dialogue system and evaluation. The usage of LLMs is expected to assist NPCs understand and respond to user inputs in a more lifelike manner, increasing realism and engagement in the VR environment. To ensure that users and NPCs interact smoothly, the system includes extensive speech recognition, natural language processing, and text-to-speech functions. Collaboration with the PETRONAS Leadership Centre on the project provides industry insights and ensures that the curriculum meets genuine requirements. The purpose of the project is to provide professionals with a realistic and efficient training tool to improve their communication skills in a controlled context.