Robust Multimodal Fusion for Human Activity Recognition

Sensor data streams are intermittent and noisy in real-world settings. This is primarily because sensors are used in various conditions The proliferation of IoT and mobile devices equipped with heterogeneous and environments without (re)calibration and proper protection, sensors has enabled new applications that rely on the which makes them susceptible to offsets and drifts [23], fusion of time-series data generated by multiple sensors with different in addition to dislocation, deformation, occlusion, and dirt/dust modalities. While there are promising deep neural network buildup [18]. For example, while the total offset and scaling error architectures for multimodal fusion, their performance falls apart of most IMUs, including LSM9DS1 manufactured by STMicroelectronics quickly in the presence of consecutive missing data and noise across and BNO055 by Bosch Sensortec, is within 1%, this error multiple modalities/sensors, the issues that are prevalent in realworld will be much higher if the sensor is not dynamically calibrated in settings. We propose Centaur, a multimodal fusion model the environment. Moreover, wireless sensors often send data to for human activity recognition (HAR) that is robust to these data a node that has enough compute power to run the fusion model.