Orthogonal Transformer: An Efficient Vision Transformer Backbone with Token Orthogonalization

Herein we provide the proof of Theorem 1 in the main text. Four lemmas with their proofs are given in advance, and the proof of Theorem 1 is in the last. Lemma A.2 Given any two non-zero vectors x and y with the same 2-norm, there exists a Householder transformation H satisfying Hx = y. Proof A.2 We can construct the Householder matrix with vector u = . Proof A.3 The matrix A can be written as a block form A = [a R = QR (8) Lemma A.4 If a n n matrix A is not only upper triangular but also orthogonal, then A is a diagonal matrix.