Announcing NVIDIA Jarvis, Combining Speech, Vision and Other Sensors into One AI SDK - NVIDIA Developer News Center

At Mobile World Congress Los Angeles, NVIDIA Co-founder and CEO Jensen Huang announced in his keynote – NVIDIA Jarvis – an SDK for building and deploying AI applications that fuse vision, speech, and other sensors into one system. NVIDIA Jarvis offers a comprehensive workflow to build, train and deploy AI systems that uses speech and visual cues such as gestures and gaze in context. For example, lip movement can be fused with speech input to identify the active speaker. Gaze can be used to understand if the speaker is engaging the AI agent or other people in the scene. This enables simultaneous multi-user, multi-context conversations with the AI system that need a deeper understanding of the context.