Adder Attention for Vision Transformer

The detailed description of the Adder Transformer is given in Figure. Red arrows indicate vector-wise operations. Then, adding an identity matrix will lead to a change of the curve of Adder Transformer in Figure.