Tiny Machine Learning (TinyML) is a novel research field aiming at integrating Machine Learning (ML) within embedded devices with limited memory, computation, and energy. Recently, a new branch of TinyML has emerged, focusing on integrating ML directly into the sensors to further reduce the power consumption of embedded devices. Interestingly, despite their state-of-the-art performance in many tasks, none of the current solutions in the literature aims to optimize the implementation of Convolutional Neural Networks (CNNs) operating directly into sensors. In this paper, we introduce for the first time in the literature the optimized design and implementation of Depth-First CNNs operating on the Intelligent Sensor Processing Unit (ISPU) within an Inertial Measurement Unit (IMU) by STMicroelectronics. Our approach partitions the CNN between the ISPU and the microcontroller (MCU) and employs an Early-Exit mechanism to stop the computations on the IMU when enough confidence about the results is achieved, hence significantly reducing power consumption. When using a NUCLEO-F411RE board, this solution achieved an average current consumption of 4.8 mA, marking an 11% reduction compared to the regular inference pipeline on the MCU, while having equal accuracy.

The SENSOR TEST 2022 conference took place last month in Nuremberg, Germany, where attendees got to check out the ISM330IS, the first sensor with an intelligent sensor processing unit (ISPU). ST announced the technology in early 2022, and its demonstration is highly symbolic as it testifies to its soon availability. In a nutshell, the ISPU is a C-programmable embedded digital signal processor (DSP) capable of running machine learning and deep learning algorithms. It is thus the next evolution of AI at the Edge, or what ST calls "The Onlife Era". Indeed, the ISM330IS includes a floating point unit for single-bit precision computations, a first in a motion sensor.