A ImageNet Pre-training Table 1: Training settings on ImageNet classification

Both RTFormer-Slim and RTFormer-Base outperform the corresponding DDRNet variants. The self-attention used for comparison is following (12). For linformer attention, we directly give a result without hyper parameter modification. Multi-head external attention can achieve a good inference speed, which is benefit from its linear complexity and the design of sharing external parameter for multiple heads. "#Params" refers to the number of parameters.