Modeling Accurate Human Activity Recognition for Embedded Devices Using Multi-level Distillation

Human Activity Recognition (HAR) based on IMU sensors is a crucial area in ubiquitous computing. Because of the trend of deploying AI on IoT devices or smartphones, more researchers are designing different HAR models for embedded devices. Deployment of models in embedded devices can help enhance the efficiency of HAR. We propose a multi-level HAR modeling pipeline called Stage-Logits-Memory Distillation (SMLDist) for constructing deep convolutional HAR models with embedded hardware support. SMLDist includes stage distillation, memory distillation, and logits distillation. Stage distillation constrains the learning direction of the intermediate features. The teacher model teaches the student models how to explain and store the inner relationship among high-dimensional features based on Hopfield networks in memory distillation. Logits distillation builds logits distilled by a smoothed conditional rule to preserve the probability distribution and enhance the softer target accuracy. We compare the accuracy, F1 macro score, and energy cost on embedded platforms of a MobileNet V3 model built by SMLDist with various state-of-the-art HAR frameworks. The product model has a good balance with robustness and efficiency. SMLDist can also compress models with a minor performance loss at an equal compression ratio to other advanced knowledge distillation methods on seven public datasets.