Paper review: Patch-based inference for TinyML

Tiny deep learning on microcontroller units (MCUs) is challenging due to the limited memory size. Memory bottleneck exists with MCUs because of the imbalanced memory distribution in convolutional neural network (CNN) designs. For instance, in MobileNetV2 only the first 5 blocks have a high peak memory ( 450kB), becoming the memory bottleneck of the entire network. The remaining 13 blocks have a low memory usage, which can easily fit a 256kB MCU. The peak memory of the initial memory-intensive stage is 8 times higher than the rest of the network.