Now in its 37th year, ICML (The International Conference on Machine Learning) is known for bringing cutting-edge research on all aspects of machine learning to the fore. This year, 1088 papers have been accepted from 4990 submissions. Here are a few interesting works to look at ICML 2020, which will be held between 13th and 18th of July. Meta-learning relies on deep networks, which makes batch normalization an essential component of meta-learning pipelines. However, there are several challenges that can render conventional batch normalization ineffective, giving rise to the need to rethink normalization in this setting.

Modern meta-learning approaches for image classification rely on increasingly deep networks to achieve state-of-the-art performance, making batch normalization an essential component of meta-learning pipelines. However, the hierarchical nature of the meta-learning setting presents several challenges that can render conventional batch normalization ineffective, giving rise to the need to rethink normalization in this setting. We evaluate a range of approaches to batch normalization for meta-learning scenarios, and develop a novel approach that we call TaskNorm. Experiments on fourteen datasets demonstrate that the choice of batch normalization has a dramatic effect on both classification accuracy and training time for both gradient based and gradient-free meta-learning approaches. Importantly, TaskNorm is found to consistently improve performance. Finally, we provide a set of best practices for normalization that will allow fair comparison of meta-learning algorithms.