A Additional Details for D2C A.1 Training diffusion models

We refer to the reader to these two papers for more details. On the high-level, this is the integration of three objectives: the reconstruction objective via the autoencoder, the diffusion objective over the latent space, and the contrastive objective over the latent space. In order to perform few-shot conditional generation, we need to implement line 4 in Algorithm 1, where an unnormalized (energy-based) model is defined over the representations. This procedure is describe in Algorithm 4. 19 Algorithm 4 Generate from labels The results are not particularly sensitive to how the discretization steps are chosen. Theorem 3. (formal) Suppose that x R B.2 D2 models address latent posterior mismatch in V AEs Theorem 2. C.1 Architecture details and hyperparameters used for training Additional details about the hyperparameters used are provided in Table 5. C.2 Additional details for conditional generation The reward per task is kept as 0.25$.