Existing MIPL algorithms often overlook the margins for attention scores andpredicted probabilities, leading tosuboptimal generalization performance.

Recent studies [42,56]have visualized the mean attention distance of ViTs, offering deeper insights into weight redundancy among attention heads across different layers.

More than twenty years after its introduction, Annealed Importance Sampling (AIS) remains one of the most effective methods for marginal likelihood estimation.