Broader Impacts NaViT enables training of vision transformers on variable size inputs, which has a profound impact on advancing adaptive computation research. By training models to handle various input size, we can explore adaptive computation techniques that dynamically adjust the computational resources based on the specific requirements of a given input. This flexibility opens up new avenues for implementing ideas that aim at adjusting allocation of compute and improving efficiency in vision tasks per input. Furthermore, NaViT computational efficiency unlocks the potential for scaling up pre-training of vision models. With the ability to handle different resolutions, models can effectively tackle more complex and diverse visual data, allowing for the development of larger and more powerful vision models.

The ubiquitous and demonstrably suboptimal choice of resizing images to a fixed resolution before processing them with computer vision models has not yet been successfully challenged.

A.1 Construction of a D-EquiStatic graph A practical method to construct a D-EquiStatic weight matrix W is provided in Alg. 3. We should mention that the "while" loop in the algorithm is adopted to guarantee kΠWk2 ρ. Construct W by (3); end Output: The D-EquiStatic weight matrix W and its associated basis indices {ut}Mt=1 A.2 Proof of Theorem 1 Before showing properties of W defined by (3), we provide two lemmas as follows. Referring to Theorem 1.6 of [32], we have the following result for a sequence of random matrices. Lemma 1 (Matrix Bernstein) Consider a sequence of K independent random n n matrices {Mi}Ki=1. Assume that each random matrix satisfies E[Mi] = 0, and kMik2 R almost surely. Theorem 1 (Formal restatement of Theorem 1) Let A(u) be defined by (2) for any u [n 1]and the D-EquiStatic weight matrix W be constructed by (3) with {ui}Mi=1 following an independent and identical uniform distribution from [n 1].

Decentralized optimization is an emerging paradigm in distributed learning in which agents achieve network-wide solutions by peer-to-peer communication without the central server. Since communication tends to be slower than computation, when each agent communicates with only a few neighboring agents per iteration, they can complete iterations faster than with more agents or a central server. However, the total number of iterations to reach a network-wide solution is affected by the speed at which the agents' information is "mixed" by communication. We found that popular communication topologies either have large maximum degrees (such as stars and complete graphs) or are ineffective at mixing information (such as rings and grids). To address this problem, we propose a new family of topologies, EquiTopo, which has an (almost) constant degree and a network-size-independent consensus rate that is used to measure the mixing efficiency. In the proposed family, EquiStatic has a degree of Θ(ln(n)), where nis the network size, and a series of time-dependent one-peer topologies, EquiDyn, has a constant degree of 1. We generate EquiDyn through a certain random sampling procedure. Both of them achieve an n-independent consensus rate. We apply them to decentralized SGD and decentralized gradient tracking and obtain faster communication and better convergence, theoretically and empirically.

In this comparison, we apply meta learning to all available networks, and show results after 10 minutes of training. The methods which use the RGB pixels produce sharp results, as the encoder and decoder do not need to be fine-tuned, but are not robust to errors in the geometry, as highlighted in the figure. These areas are inpainted in the version with a decoder.