Second, on channel aspect, representation exhibits diversity ondifferent channels. But the scarce data can not enable ViTs to learn strong enough representation for accurate recognition.
Regularization terms, suchasL2 regularization leading toweightdecay,arecommonly employed during training to improve a model's generalization capabilities.