A Novel Task-Driven Diffusion-Based Policy with Affordance Learning for Generalizable Manipulation of Articulated Objects

Abstract--Despite recent advances in dexterous manipulations, the manipulation of articulated objects and generalization across different categories remain significant challenges. T o address these issues, we introduce DART, a novel framework that enhances a d iffusion-based policy with a ffor dance learning and linear t emporal logic (L TL) representations to improve the learning efficiency and generalizability of articulated dexterous manipulation. Specifically, DART leverages L TL to understand task semantics and affordance learning to identify optimal interaction points. Additionally, we exploit an optimization method based on interaction data to refine actions, overcoming the limitations of traditional diffusion policies that typically rely on offline reinforcement learning or learning from demonstrations. Experimental results demonstrate that DART outperforms most existing methods in manipulation ability, generalization performance, transfer reasoning, and robustness. The manipulation of articulated objects has been an interesting and important topic in robotic learning. Although prior research has demonstrated promising results in the manipulation of rigid bodies, significant challenges persist when it comes to handling articulated objects [1]. Generalizing to various types of articulated objects [2] is particularly difficult for dexterous manipulations. For example, if a dexterous hand can open the lid of a toilet, it should also be capable of opening the lid of a garbage can, despite their cosmetic differences. While many recent efforts have focused on improving the robotic generalization performance [3] or reducing the exploration burden [4], enhancing the learning efficiency or improve the generalization ability for high degrees of freedom (DOF) skills, such as dexterous manipulation, remains a challenging problem, not to mention achieving both simultaneously.