One Network to Fit All Hardware: New MIT AutoML Method Trains 14X Faster Than SOTA NAS

AI is now integrated into countless scenarios, from tiny drones to huge cloud platforms. Every hardware platform is ideally paired with a tailored AI model that perfectly meets requirements in terms of performance, efficiency, size, latency, etc. However even a single model architecture type needs tweaking when applied to different hardware, and this requires researchers spend time and money training them independently. Popular solutions today include either designing models specialized for mobile devices or pruning a large network by reducing redundant units, aka model compression. A group of MIT researchers (Han Cai, Chuang Gan and Song Han) have introduced a "Once for All" (OFA) network that achieves the same or better level accuracy as state-of-the-art AutoML methods on ImageNet, with a significant speedup in training time. A major innovation of the OFA network is that researchers don't need to design and train a model for each scenario, rather they can directly search for an optimal subnetwork using the OFA network.