Google Open-Sources Computer Vision Model Big Transfer

Google Brain has released the pre-trained models and fine-tuning code for Big Transfer (BiT), a deep-learning computer vision model. The models are pre-trained on publicly-available generic image datasets and can meet or exceed state-of-the-art performance on several vision benchmarks after fine-tuning on just a few samples. Paper co-authors Lucas Beyer and Alexander Kolesnikov gave an overview of their work in a recent blog post. To help advance the performance of deep-learning vision models, the team investigated large-scale pre-training and the effects of model size, dataset size, training duration, normalization strategy, and hyperparameter choice. As a result of this work, the team developed a "recipe" of components and training heuristics that achieves strong performance on a variety of benchmarks, including an "unprecedented top-5 accuracy of 80.0%" on the ObjectNet dataset.