Leveraging the compositional nature of our world to expedite learning and facilitate generalization is a hallmark of human perception. In machine learning, on the other hand, achieving compositional generalization has proven to be an elusive goal, even for models with explicit compositional priors. To get a better handle on compositional generalization, we here approach it from the bottom up: Inspired by identifiable representation learning, we investigate compositionality as a property of the data-generating process rather than the data itself. This reformulation enables us to derive mild conditions on only the support of the training distribution and the model architecture, which are sufficient for compositional generalization. We further demonstrate how our theoretical framework applies to real-world scenarios and validate our findings empirically. Our results set the stage for a principled theoretical study of compositional generalization.

We present the first diffusion-based framework that can learn an unknown distribution using only highly-corrupted samples. This problem arises in scientific applications where access to uncorrupted samples is impossible or expensive to acquire. Another benefit of our approach is the ability to train generative models that are less likely to memorize any individual training sample, since they never observe clean training data. Our main idea is to introduce additional measurement distortion during the diffusion process and require the model to predict the original corrupted image from the further corrupted image. We prove that our method leads to models that learn the conditional expectation of the full uncorrupted image given this additional measurement corruption. This holds for any corruption process that satisfies some technical conditions (and in particular includes inpainting and compressed sensing). We train models on standard benchmarks (CelebA, CIFAR-10 and AFHQ) and show that we can learn the distribution even when all the training samples have 90%of their pixels missing. We also show that we can finetune foundation models on small corrupted datasets (e.g. MRI scans with block corruptions) and learn the clean distribution without memorizing the training set.

Neural Information Processing Systems http://nips.cc/

Motion Compensation We compare our method to the traditional motion-compensated coding378 approach that forms the core of inter-picture coding in well established compression standards such379 as MPEG. Block matching is an essential component of these standards, allowing the compression of380 video content by up to three orders of magnitude with moderate loss of information. For each block381 in a frame, typical coders search for the most similar spatially displaced block in the previous frame382 (typically measured with MSE), and communicate the displacement coordinates to allow prediction383 of frame content by translating blocks of the (already transmitted) previous frame. We implemented384 a "diamond search" algorithm [29] operating on blocks of 8 8 pixels, with a maximal search385 distance of 8 pixels which balances accuracy of motion estimates and speed of estimation (the search386 step is computationally intensive). We use the estimated displacements to perform causal motion387 compensation (cMC), using displacement vectors estimated from the previous two observed frames388 (xt 1 and xt) to predict the next frame (xt+1) rather than the current one (as in MPEG).389

Score-based diffusion models (SBDMs) have achieved state-of-the-art results in image generation. In this paper, we propose a Non-isotropic Gaussian Diffusion Model (NGDM) for image editing, which requires editing the source image while preserving the image regions irrelevant to the editing task. We construct NGDM by adding independent Gaussian noises with different variances to different image pixels.

Figure 10: Visual results on Pascal validation set with 60 labels. As is shown, ARCO-SGand ARCO-SAG consistently yield more accurate and sharper boundaries compared to all other SSL methods.