CoAtNet: MarryingConvolutionandAttention forAllDataSizes

However, these approaches are either ad-hoc or focused on injecting a particular property, lacking a systematic understanding of the respective roles of convolution and attentionwhencombined.