A Appendix

A.1 Remarks on executed benchmarks We executed all benchmarks faithfully and to the best of our knowledge. In particular, with regards to the multi-modal transformer scaling behavior, as there are in fact no such studies for AR models yet to compare to. The integration of Chefer was straightforward. We did not further investigate or fine-tune evaluations to any method. In Figure 1 we ran Chefer with a full backward pass.