Mixup refers to interpolation-based data augmentation, originally motivated as a way to go beyond empirical risk minimization (ERM). Its extensions mostly focus on the definition of interpolation and the space (input or embedding) where it takes place, while the augmentation process itself is less studied. In most methods, the number of generated examples is limited to the mini-batch size and the number of examples being interpolated is limited to two (pairs), in the input space. We make progress in this direction by introducing MultiMix, which generates an arbitrarily large number of interpolated examples beyond the mini-batch size, and interpolates the entire mini-batch in the embedding space.

A.1 More on setup Settings and hyperparameters We train MultiMix and Dense MultiMix with mixed examples only. We use a mini-batch of size b = 128 examples in all experiments. Following Manifold Mixup [ 51 ], for every mini-batch, we apply MultiMix with probability 0 . For multi-GPU experiments, all training hyperparameters including m and n are per GPU. For Dense MultiMix, the spatial resolution is r =4 4 = 16 on CIFAR-10/100 and r =7 7 = 49 on Imagenet by default.

Mixup refers to interpolation-based data augmentation, originally motivated as a way to go beyond empirical risk minimization (ERM). Its extensions mostly focus on the definition of interpolation and the space (input or embedding) where it takes place, while the augmentation process itself is less studied. In most methods, the number of generated examples is limited to the mini-batch size and the number of examples being interpolated is limited to two (pairs), in the input space. We make progress in this direction by introducing MultiMix, which generates an arbitrarily large number of interpolated examples beyond the mini-batch size, and interpolates the entire mini-batch in the embedding space.

Mixup refers to interpolation-based data augmentation, originally motivated as a way to go beyond empirical risk minimization (ERM). Its extensions mostly focus on the definition of interpolation and the space (input or feature) where it takes place, while the augmentation process itself is less studied. In most methods, the number of generated examples is limited to the mini-batch size and the number of examples being interpolated is limited to two (pairs), in the input space. We make progress in this direction by introducing MultiMix, which generates an arbitrarily large number of interpolated examples beyond the mini-batch size and interpolates the entire mini-batch in the embedding space. Effectively, we sample on the entire convex hull of the mini-batch rather than along linear segments between pairs of examples. On sequence data, we further extend to Dense MultiMix. We densely interpolate features and target labels at each spatial location and also apply the loss densely. To mitigate the lack of dense labels, we inherit labels from examples and weight interpolation factors by attention as a measure of confidence. Overall, we increase the number of loss terms per mini-batch by orders of magnitude at little additional cost. This is only possible because of interpolating in the embedding space. We empirically show that our solutions yield significant improvement over state-of-the-art mixup methods on four different benchmarks, despite interpolation being only linear. By analyzing the embedding space, we show that the classes are more tightly clustered and uniformly spread over the embedding space, thereby explaining the improved behavior.

In this article, I will discuss a new semi-supervised, multi-tasking medical imaging method called MultiMix, by Ayaan Haque (me), Abdullah-Al-Zubaer Imran, Adam Wang, and Demetri Terzopoulos. Our paper was accepted to ISBI 2021 in the full-paper track and was presented at the conference in April. The extension of our paper with improved results was published in the MELBA Journal as well. This article will cover a review of the methods, results, and a short code review. The code is available here. MultiMix performs joint semi-supervised classification and segmentation by employing a confidence-based augmentation strategy and a novel saliency bridge module which provides explainability for the joint tasks. Deep learning-based models, when fully-supervised can be efficient in performing complex image analysis tasks, but this performance relies heavily upon the availability of large labeled datasets. Especially in the medical imaging domain, labels are expensive, time-consuming, and prone to observer variations.