Recently there has been significant theoretical progress on understanding the convergence andgeneralization ofgradient-based methods onnonconvexlosses withoverparameterized models. Nevertheless, manyaspectsofoptimization and generalization and in particular the critical role of small random initialization are not fully understood.

Recently there has been significant theoretical progress on understanding the convergence andgeneralization ofgradient-based methods onnonconvexlosses withoverparameterized models. Nevertheless, manyaspectsofoptimization and generalization and in particular the critical role of small random initialization are not fully understood.

These authors contributed equally to this work 38th Conference on Neural Information Processing Systems (NeurIPS 2024).

Adepiction 4. The architecture 2. Ourf is

Vanilla training canbeconsidered asaspecial case where no perturbation is allowed, i.e., zero adversarial budget. Therefore, we focus on the impact of the adversarial budget size on the loss landscape.

Existing image-text modality alignment in Vision Language Models (VLMs) treats each text token equally in an autoregressive manner.