Gaze estimation is a valuable technology with numerous applications in fields such as human-computer interaction, virtual reality, and medicine. This report presents the implementation of a gaze estimation system using the Sony Spresense microcontroller board and explores its performance in latency, MAC/cycle, and power consumption. The report also provides insights into the system's architecture, including the gaze estimation model used. Additionally, a demonstration of the system is presented, showcasing its functionality and performance. Our lightweight model TinyTrackerS is a mere 169Kb in size, using 85.8k parameters and runs on the Spresense platform at 3 FPS.

Intelligent edge vision tasks encounter the critical challenge of ensuring power and latency efficiency due to the typically heavy computational load they impose on edge platforms.This work leverages one of the first "AI in sensor" vision platforms, IMX500 by Sony, to achieve ultra-fast and ultra-low-power end-to-end edge vision applications. We evaluate the IMX500 and compare it to other edge platforms, such as the Google Coral Dev Micro and Sony Spresense, by exploring gaze estimation as a case study. We propose TinyTracker, a highly efficient, fully quantized model for 2D gaze estimation designed to maximize the performance of the edge vision systems considered in this study. TinyTracker achieves a 41x size reduction (600Kb) compared to iTracker [1] without significant loss in gaze estimation accuracy (maximum of 0.16 cm when fully quantized). TinyTracker's deployment on the Sony IMX500 vision sensor results in end-to-end latency of around 19ms. The camera takes around 17.9ms to read, process and transmit the pixels to the accelerator. The inference time of the network is 0.86ms with an additional 0.24 ms for retrieving the results from the sensor. The overall energy consumption of the end-to-end system is 4.9 mJ, including 0.06 mJ for inference. The end-to-end study shows that IMX500 is 1.7x faster than CoralMicro (19ms vs 34.4ms) and 7x more power efficient (4.9mJ VS 34.2mJ)

TensorFlow is Google's open-source software library for machine learning that developers and researchers can use for a range of application areas. The well-known and widely spread library comes in different flavors ready to run on various operating systems and hardware. Specifically, the TensorFlow Lite for Microcontrollers (TFLM) version is designed to run on microcontroller systems where the hardware resources are more limited compared to larger computerized systems. The footprint of TFLM is typically in the order of only 10's of kB. Now you can develop solutions with TensorFlow for the Spresense microcontroller board from Sony.

We're happy to announce that the cloud-based AI tool Edge Impulse now supports Sony's Spresense microcontroller board. It's free to start using Edge Impulse and it simplifies the work with training and deploying AI models, including the data collection. With Spresense and Edge Impulse, you get an end-to-end solution for industrial machine learning with ultra-low power consumption and high computing capability. With Spresense's state-of-the-art visual and hi-res audio features along its 6-core microcontroller and a range of sensor add-on options, you have a powerful edge device that you can train using the graphical user interface of Edge Impulse. In a simple manner of dragging, dropping and clicking, you step your way through the machine learning process.