Transformer in Transformer

Transformer is a type of neural network mainly based on self-attention mechanism . Transformer is widely used in the field of natural language processing (NLP), e.g., the famous BERT and GPT3 models. Inspired by the breakthrough of transformer in NLP, researchers have recently applied transformer to computer vision (CV) tasks such as image recognition, object detection, and image processing . For example, DETR treats object detection as a direct set prediction problem and solve it using a transformer encoder-decoder architecture. Compared to the mainstream CNN models, these transformer-based models have also shown promising performance on visual tasks .