Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation

Our approach integrates state estimation and dynamics modeling under a consistent architecture and training paradigm. Our diffusion-based perception model generates cloth states from partial observations, and the diffusion-based dynamics model generates physically plausible future states conditioned on action sequences, enabling robust model-based control. Our work demonstrates the potential of diffusion models in state estimation and dynamics modeling for manipulation tasks involving partial observability and complex dynamics. Abstract--Manipulating deformable objects like cloth is challenging states given the current state and robot actions. Leveraging a due to their complex dynamics, near-infinite degrees of transformer-based diffusion model, our method achieves highfidelity freedom, and frequent self-occlusions, which complicate state state reconstruction while reducing long-horizon dynamics estimation and dynamics modeling. Prior work has struggled with prediction errors by an order of magnitude compared to robust cloth state estimation, while dynamics models, primarily GNN-based approaches. Integrated with model-predictive control based on Graph Neural Networks (GNNs), are limited by their (MPC), our framework successfully executes cloth folding on a locality. Inspired by recent advances in generative models, we real robotic system, demonstrating the potential of generative hypothesize that these expressive models can effectively capture models for manipulation tasks with partial observability and intricate cloth configurations and deformation patterns from complex dynamics.