multi-model
Multi-models with averaging in feature domain for non-invasive blood glucose estimation
Wei, Yiting, Ling, Bingo Wing-Kuen, Liu, Qing, Liu, Jiaxin
Diabetes is a serious chronic metabolic disease. In the recent years, more and more consumer technology enterprises focusing on human health are committed to implementing accurate and non-invasive blood glucose algorithm in their products. However, due to the interference from the external environment, these wearable non-invasive methods yield the low estimation accuracy. To address this issue, this paper employs different models based on different ranges of the blood glucose values for performing the blood glucose estimation. First the photoplethysmograms (PPGs) are acquired and they are denoised via the bit plane singular spectrum analysis (SSA) method. Second, the features are extracted. For the data in the training set, first the features are averaged across the measurements in the feature domain via the optimization approach. Second, the random forest is employed to sort the importance of each feature. Third, the training set is divided into three subsets according to the reference blood glucose values. Fourth, the feature vectors and the corresponding blood glucose values in the same group are employed to build an individual model. Fifth, for each feature, the average of the feature values for all the measurements in the same subset is computed. For the data in the test set, first, the sum of the weighted distances between the test feature values and the average values obtained in the above is computed for each model. Here, the weights are defined based on the importance sorted by the random forest obtained in the above. The model corresponding to the smallest sum is assigned. Finally, the blood glucose value is estimated based on the corresponding model. Compared to the state of arts methods, our proposed method can effectively improve the estimation accuracy.
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings
Rotem, Daniel, Hassid, Michael, Mamou, Jonathan, Schwartz, Roy
Adaptive inference is a simple method for reducing inference costs. The method works by maintaining multiple classifiers of different capacities, and allocating resources to each test instance according to its difficulty. In this work, we compare the two main approaches for adaptive inference, Early-Exit and Multi-Model, when training data is limited. First, we observe that for models with the same architecture and size, individual Multi-Model classifiers outperform their Early-Exit counterparts by an average of 2.3%. We show that this gap is caused by Early-Exit classifiers sharing model parameters during training, resulting in conflicting gradient updates of model weights. We find that despite this gap, Early-Exit still provides a better speed-accuracy trade-off due to the overhead of the Multi-Model approach. To address these issues, we propose SWEET (Separating Weights in Early Exit Transformers), an Early-Exit fine-tuning method that assigns each classifier its own set of unique model weights, not updated by other classifiers. We compare SWEET's speed-accuracy curve to standard Early-Exit and Multi-Model baselines and find that it outperforms both methods at fast speeds while maintaining comparable scores to Early-Exit at slow speeds. Moreover, SWEET individual classifiers outperform Early-Exit ones by 1.1% on average. SWEET enjoys the benefits of both methods, paving the way for further reduction of inference costs in NLP.