Supplementary Materials - Adaptive Online Replanning with Diffusion Models

In the supplementary, we first discuss the experimental details and hyperparameters in Section A. Next, we analyze the impact of different numbers of diffusion steps N on the replanning process in Section B, and further present the visualization in RLBench in Section C. Finally, we discuss how to compute the likelihood in Section D. In detail, our architecture comprises a temporal U-Net structure with six repeated residual networks. Each network consists of two temporal convolutions followed by GroupNorm [6], and a final Mish nonlinearity [4]. Additionally, We incorporate timestep and conditions embeddings, which are both 128-dimensional vectors produced by MLP, within each block. The probability ϵ of random actions is set to 0.03 in Stochastic Environments. The total number of diffusion steps, corresponding to the number of diffusion steps for Replan from scratch is set to 256 in Maze2D, 200 in Stochastic Environments, and 400 in RLBench.