SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation