We present BoTNet, a conceptually simple yet powerful backbone architecture that incorporates self-attention for multiple computer vision tasks including image classification, object detection and instance segmentation. By just replacing the spatial convolutions with global self-attention in the final three bottleneck blocks of a ResNet and no other changes, our approach improves upon the baselines significantly on instance segmentation and object detection while also reducing the parameters, with minimal overhead in latency. Through the design of BoTNet, we also point out how ResNet bottleneck blocks with self-attention can be viewed as Figure 1: Left: A ResNet Bottleneck Block, Right: A Bottleneck Transformer blocks. Without any bells and whistles, BoTNet Transformer (BoT) block. The only difference is achieves 44.4% Mask AP and 49.7% Box AP on the COCO the replacement of the spatial 3 3 convolution layer with Instance Segmentation benchmark using the Mask R-CNN Multi-Head Self-Attention (MHSA). The structure of the framework; surpassing the previous best published single self-attention layer is described in Figure 4. model and single scale results of ResNeSt [72] evaluated on the COCO validation set. Finally, we present a simple adaptation of the BoTNet design for image classification, from a large neighborhood can be useful in learning relationships resulting in models that achieve a strong performance of across objects [34]. In order to globally aggregate the 84.7% top-1 accuracy on the ImageNet benchmark while locally captured filter responses, convolution based architectures being up to 2.33x faster in "compute"