Deep learning inference on embedded devices is a burgeoning field with myriad applications because tiny embedded devices are omnipresent. But we must overcome major challenges before we can benefit from this opportunity. Embedded processors are severely resource constrained. Their nearest mobile counterparts exhibit at least a 100---1,000x difference in compute capability, memory availability, and power consumption. As a result, the machine-learning (ML) models and associated ML inference framework must not only execute efficiently but also operate in a few kilobytes of memory. Also, the embedded devices' ecosystem is heavily fragmented. To maximize efficiency, system vendors often omit many features that commonly appear in mainstream systems, including dynamic memory allocation and virtual memory, that allow for cross-platform interoperability. The hardware comes in many flavors (e.g., instruction-set architecture and FPU support, or lack thereof). We introduce TensorFlow Lite Micro (TF Micro), an open-source ML inference framework for running deep-learning models on embedded systems. TF Micro tackles the efficiency requirements imposed by embedded-system resource constraints and the fragmentation challenges that make cross-platform interoperability nearly impossible. The framework adopts a unique interpreter-based approach that provides flexibility while overcoming these challenges. This paper explains the design decisions behind TF Micro and describes its implementation details. Also, we present an evaluation to demonstrate its low resource requirement and minimal run-time performance overhead.

We are going to the AIoT Summit on December 3rd with Ambiq Micro and TensorFlow, and we wanted to give you a sneak peek! On December 3rd, you will find SparkFun CTO Kirk Bennel and Creative Technologist Rob Reynolds at the AIoT Dev Summit! In collaboration with Ambiq Micro and TensorFlow, we are going to be running a hands-on workshop where you will be able to build your very own Harry Potter wand! That's right, all of us muggles out there can finally get a magic wand without ever having to break into Ollivanders like a certain dark wizard. This is an experiential and experimental workshop that focuses on the use of TensorFlow Lite on a low-power microcontroller to perform machine learning.

This post was originally published by Sandeep Mistry and Dominic Pajak on the TensorFlow blog. Arduino is on a mission to make machine learning simple enough for anyone to use. We've been working with the TensorFlow Lite team over the past few months and are excited to show you what we've been up to together: bringing TensorFlow Lite Micro to the Arduino Nano 33 BLE Sense. In this article, we'll show you how to install and run several new TensorFlow Lite Micro examples that are now available in the Arduino Library Manager. The first tutorial below shows you how to install a neural network on your Arduino board to recognize simple voice commands.

This post was originally published by Sandeep Mistry and Dominic Pajak on the TensorFlow blog. Arduino is on a mission to make machine learning simple enough for anyone to use. We've been working with the TensorFlow Lite team over the past few months and are excited to show you what we've been up to together: bringing TensorFlow Lite Micro to the Arduino Nano 33 BLE Sense. In this article, we'll show you how to install and run several new TensorFlow Lite Micro examples that are now available in the Arduino Library Manager. The first tutorial below shows you how to install a neural network on your Arduino board to recognize simple voice commands.

Over the next decade, embedded is going to experiencing the kind of innovation we haven't seen since the late 2000s when open wireless, protocols and cryptography (and as a result, 32-bit MCUs) were introduced. Today most people think about Machine Learning as highly complex, large, and extremely memory and compute hungry -- with clusters of GPUs/TPUs heating whole towns... Now the age of tinyML has come -- we can already run meaningful ML inference on Cortex-M equivalent hardware. Rapid improvements in modern 32-bit MCU compute power efficiency and math capabilities (FPU, vector extensions), together with advancements in neural operators, architecture and quantization along with better open source tooling like TensorFlow Lite Micro are making this possible. For example, we recently built a complete DSP, Anomaly Detection and NN classifier for complex events on real-time 3-axis accelerometer data in software on a standard Cortex-M4 in just 6.6 kB of RAM and 20 kB of Flash. We are experiencing the start of what I call the "3rd wave of embedded compute."