To improve how sporting events are covered in the news, a new AI 3D pose estimation model was recently developed by a group of researchers from The New York Times R&D group, and wrnch, an AI-computer vision company and member of the NVIDIA Inception program. The 3D pose estimation model can help extract data the human eye can easily miss and help journalists tell a story with more concrete data. "Traditional motion capture techniques require an athlete to wear physical markers. But this isn't possible during live sporting events. Instead, we built a solution that uses our photographers' cameras, machine learning and computer vision to capture this data as an event unfolds," the researchers stated in their article, Estimating 3D Poses of Athletes at Live Sporting Events.

Editor's note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Safety Force Field (SFF) vehicle software is designed specifically for collision avoidance. It acts as an independent supervisor on the actions of the vehicle's primary planning and control system, which could be either human-driven or autonomous. Specifically, SFF performs real-time double-checks of the controls that were chosen by the primary system. If SFF deems the controls to be unsafe, it will veto and correct the primary system's decision.

To help autonomous vehicles and robots potentially spot objects that lie just outside a system's direct line-of-sight, Stanford, Princeton, Rice, and Southern Methodist universities researchers developed a deep learning-based system that can detect objects, including words and symbols, around corners. "Compared to other approaches, our non-line-of-sight imaging system provides uniquely high resolutions and imaging speeds," said Stanford University's Chris Metzler, on the Rice University post, Cameras see around corners in real time with deep learning. "These attributes enable applications that wouldn't otherwise be possible," he added. To achieve this, the system relies on a laser that can capture detailed images of objects around corners in real time. Specifically, a light from a high-speed laser is beamed onto a wall, the light from the hidden area bounces back to the wall, and that light is reflected to a camera.

Navigating a new indoor space without any prior knowledge or even a map is a challenging task for a human, let alone a robot. To help develop intelligent machines that interact more effectively with complex 3D environments, Facebook researchers developed a GPU-accelerated deep reinforcement learning model that achieves near 100 percent success in navigating a variety of virtual environments without a pre-provided map. To achieve this breakthrough, the team focused their work on developing an efficient approach to scaling RL models, which require a significant number of training samples, using multi-node distribution. "A single parameter server and thousands of (typically CPU) workers may be fundamentally incompatible with the needs of modern computer vision and robotics communities," the researchers explained in their post, Near-perfect point-goal navigation from 2.5 billion frames of experience. "Unlike Gym or Atari, 3D simulators require GPU acceleration…. The desired agents operate from high-dimensional inputs (pixels) and use deep networks, such as ResNet50, which strain the parameter server. Thus, existing distributed RL architectures do not scale and there is a need to develop a new distributed architecture."

The article below is a guest post by Nuance, a company focused on conversational AI. In this post, Nuance engineers describe their use of NVIDIA's automatic mixed precision to speed up their AI models in the healthcare industry. Nuance's ambient clinical intelligence (ACI) technology is an example of how it is accelerating development of solutions for urgent problems in the U.S. healthcare system by training its automatic speech recognition (ASR) and natural language processing (NLP) models using NVIDIA's Automatic Mixed Precision capabilities on Volta and Turing GPUs with Tensor Cores. ACI addresses what the World Medical Association calls a "pandemic of physician burnout" caused by huge amounts of electronic paperwork. Doctors spend two hours completing documentation for every hour they deliver care.

Today, NVIDIA posted the fastest results on new MLPerf benchmarks measuring the performance of AI inference workloads in data centers and at the edge. The new results come on the heels of the company's equally strong results in the MLPerf benchmarks posted earlier this year. MLPerf's five inference benchmarks -- applied across a range of form factors and four inferencing scenarios -- cover such established AI applications as image classification, object detection and translation. NVIDIA topped all five benchmarks for both data center-focused scenarios (server and offline), with Turing GPUs providing the highest performance per processor among commercially available entries. Xavier provided the highest performance among commercially available edge and mobile SoCs under both edge-focused scenarios (single-stream and multistream). All of NVIDIA's MLPerf results were achieved using NVIDIA TensorRT 6 high-performance deep learning inference software that optimizes and deploys AI applications easily in production from the data center to the edge.

At Mobile World Congress Los Angeles, NVIDIA Co-founder and CEO Jensen Huang announced in his keynote – NVIDIA Jarvis – an SDK for building and deploying AI applications that fuse vision, speech, and other sensors into one system. NVIDIA Jarvis offers a comprehensive workflow to build, train and deploy AI systems that uses speech and visual cues such as gestures and gaze in context. For example, lip movement can be fused with speech input to identify the active speaker. Gaze can be used to understand if the speaker is engaging the AI agent or other people in the scene. This enables simultaneous multi-user, multi-context conversations with the AI system that need a deeper understanding of the context.

Safety Force Field (SFF) vehicle software is designed specifically for collision avoidance. It acts as an independent supervisor on the actions of the vehicle's primary planning and control system, which could be either human-driven or autonomous. Specifically, SFF performs real-time double-checks of the controls that were chosen by the primary system. If SFF deems the controls to be unsafe, it will veto and correct the primary system's decision. SFF is provably safe, in the sense that, if all road participants comply with SFF and the perception and vehicle controls are within expected design margins, then it can be mathematically proven that no collisions can occur.

This summer, student interns at Booz Allen Hamilton bested the competition on edge computing with the help of NVIDIA Jetson Nano. The Booz Allen Summer Games Challenge (SGC) calls on student interns across the U.S. to develop breakthrough solutions for its clients' most pressing problems. This summer, Project RAZOR placed top 10 among artificial intelligence and machine learning projects with a fully autonomous ground vehicle powered by Nano. The team competed against 80 other teams of four to five students developing projects over 10 weeks. Previous SGC winning projects include technology that helps the blind navigate, as well as ways to fight human trafficking and global disease.

Redwood City, California-based Skydio and member of NVIDIA's startup accelerator, Inception, has just released the latest version of their AI capable GPU-accelerated drone, Skydio 2. Comprised of six 4K cameras, with an NVIDIA Jetson TX2 as the processor for the autonomous system, Skydio 2 is capable of flying for up to 23 minutes at a time and can be piloted by either an experienced pilot or by the AI-based system. The Jetson TX2 has 256 GPU cores and is capable of 1.3 trillion operations a second. According to the team, the drone uses nine custom deep neural networks that help the drone track up to 10 objects while traveling at speeds of 36 miles per hour. "Skydio 2 enables you to capture everything from a backyard pickup game to a downhill adventure with a single tap, the company wrote in blog post. "It builds on Skydio R1's foundation and takes it to the next level."