A Model Architecture

In this section, we provide comprehensive details about the Transformer model architectures considered in this work. Experiments conducted on both DMLab and RoboMimic include RGB image observations. Detailed model parameters are listed in Table A.1. We learn 16-dim embedding vectors for all discrete actions. To encode proprioceptive measurement in RoboMimic, we follow Mandlekar et al.