A Experimental details

Table 3: Experimental details for all results reported. In rows where GPU-hours are given as a range, different runs with identical settings took varying times due to varying performance of our computational infrastructure. The total compute required for this project, including all training, evaluation, and preliminary runs, was roughly 3.5 GPU-years. We used a mixture of NVIDIA RTX A5000s (on an internal cluster) and NVIDIA A100s (from a cloud provider). Due to the expensive nature of drawing samples from both FDM and our baselines, we compute all quantitative metrics reported over the first 100 videos of the test set for GQN-Mazes and MineRL.