It is targeted to those looking for an out-of-the-box solution to test the Machine Learning and Artificial Intelligence capabilities of NVIDIA Jetson boards. It offers a portable and easy-to-use environment that can be executed on any NVIDIA Jetson board. The project includes a set of example applications that exercise ML and AI frameworks such as DeepStream, OpenCV, GStreamer, GstInference, among others. Each example comes with the source code and they are implemented in different languages and using different frameworks (bash scripts, python, C/C, GStreamer, etc). You can find the example that is closer to your use case, and use it as a starting point for your demos and exploratory tests.

We propose NNStreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to deep neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI. It is to process neural networks on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signify the need for on-device AI, especially if we deploy a massive number of devices. NNStreamer efficiently handles neural networks with complex data stream pipelines on devices, significantly improving the overall performance with minimal efforts. Besides, NNStreamer simplifies implementations and allows reusing off-the-shelf media filters directly, which reduces developmental costs significantly. We are already deploying NNStreamer for a wide range of products and platforms, including the Galaxy series and various consumer electronic devices. The experimental results suggest a reduction in developmental costs and enhanced performance of pipeline architectures and NNStreamer. It is an open-source project incubated by Linux Foundation AI, available to the public and applicable to various hardware and software platforms.

NVIDIA's DeepStream SDK delivers a complete streaming analytics toolkit for AI-based video and image understanding, as well as multi-sensor processing. DeepStream is an integral part of NVIDIA Metropolis, the platform for building end-to-end services and solutions for transforming pixels and sensor data to actionable insights. The DeepStream application framework features hardware-accelerated building blocks, called plugins, that bring deep neural networks and other complex processing tasks into a stream processing pipeline. DeepStream enables real-time understanding of video and sensor data that is rich and multi-modal. The SDK uses the open source GStreamer to deliver high throughput with a low-latency streaming framework.

We propose nnstreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI; i.e., processing neural networks directly on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signifies the need for on-device AI especially when a huge number of devices with real-time data processing are deployed. Nnstreamer efficiently handles neural networks with complex data stream pipelines on devices, improving the overall performance significantly with minimal efforts. Besides, nnstreamer simplifies the neural network pipeline implementations and allows reusing off-shelf multimedia stream filters directly; thus it reduces the developmental costs significantly. Nnstreamer is already being deployed with a product releasing soon and is open source software applicable to a wide range of hardware architectures and software platforms.

The Snackbot, is a robot designed in collaboration between the Robotics Institute, and the Human Computer Interaction Institute of Carnegie Mellon University. The Snackbot was created to traverse the halls of Carnegie Mellon University, and deliver food items ordered by occupants of the offices. The goal of this development project for the Snackbot, was to refine the audio/video synchronization, and to also create a simple way to log, and stream that data over a network. Such a task requires that one not only carefully consider different pieces of software to use, but also that they can apply it across the necessary platform. For the Snackbot, the sight, and sound are important qualities, especially when testing out in the field using an operator. That ability is crucial when preparing an interactive robot to autonomously carry out its task efficiently.