Adversarially Trained Model Compression: When Robustness Meets Efficiency

As more Internet-of-Things (IoT) devices come online, they are equipped with the ability to ingest and analyze information from their ambient environments via sensor inputs. Over the past few years, convolutional neural networks (CNNs) have led to rapid advances in the predictive performance, that approach and sometimes exceed human performance in a large variety of tasks [Deng et al., 2009]. It is appealing to deploy CNNs onto IoT devices to interpret multimedia big data and intelligently react to both user and environmental events. However, the efficiency of typical CNN models (e.g., size, inference speed, and energy cost) becomes a critical hurdle. Their often prohibitive complexity remains a major inhibitor for their more extensive applications in IoT systems, that are often resource-constrained and latency-sensitive. Therefore, CNN model compression [Cheng et al., 2017] is becoming an increasingly demanded technique and has been extensively studied [Liu et al., 2015, Zhou et al., 2016, Li et al., 2016]. 1 On a separate note, the prevailing deployment of CNNs also calls for attention to their robustness.