Building upon Miller's framework of explanations derived from research in multiple

Recently,there has been agrowing interest in developing saliencymethods that providevisualexplanations ofnetworkpredictions. Still,theusability ofexisting methods is limited to image classification models. To overcome this limitation, we extend the existing approaches to generategrid saliencies, which provide spatially coherent visualexplanations for(pixel-level)denseprediction networks.

In machine learning applications, an input data example typically consists ofmanydata elements.

We propose a non-iterative method that enforces the following ideas during training.

KV cache stores key and value states from previous tokens to avoid re-computation, yet it demands substantial storage space, especially for long sequences. Adaptive KV cache compression seeks to discern the saliency of tokens, preserving vital information while aggressively compressing those of less importance.

Mixup is a commonly adopted data augmentation technique for image classification. Recent advances in mixup methods primarily focus on mixing based on saliency. However, many saliency detectors require intense computation and are especially burdensome for parameter-heavy transformer models. To this end, we propose TokenMixup, an efficient attention-guided token-level data augmentation method that aims to maximize the saliency of a mixed set of tokens. TokenMixup provides 15 faster saliency-aware data augmentation compared to gradient-based methods. Moreover, we introduce a variant of TokenMixup which mixes tokens within a single instance, thereby enabling multi-scale feature augmentation. Experiments show that our methods significantly improve the baseline models' performance on CIFAR and ImageNet-1K, while being more efficient than previous methods. We also reach state-of-the-art performance on CIFAR-100 among from-scratch transformer models.