create xr experience
Create XR Experiences Using Natural-Language Voice Commands: Test Project Mellon
Project Mellon is a lightweight Python package capable of harnessing the heavyweight power of speech AI (NVIDIA Riva) and large language models (LLMs) (NVIDIA NeMo service) to simplify user interactions in immersive environments. NVIDIA announced at NVIDIA GTC 2023 that developers can start testing Project Mellon to explore creating hands-free extended reality (XR) experiences controlled by natural-language voice commands. Words can move mountains, as J.R.R. Tolkien's riddle guarding the Doors of Durin ("Speak friend, and enter") reminds us. The fundamental idea behind Project Mellon is that the power of speech AI and LLMs can be harnessed in a practical way--to open doors, and do so much more--in the virtual world. In XR, user interfaces can be complicated and difficult to use, disrupting the sense of natural immersion that is the essence of virtual, mixed, and augmented realities.