Goto

Collaborating Authors

 joint position


Tac2Motion: Contact-Aware Reinforcement Learning with Tactile Feedback for Robotic Hand Manipulation

Kim, Yitaek, Rask, Casper Hewson, Sloth, Christoffer

arXiv.org Artificial Intelligence

This paper proposes Tac2Motion, a contact-aware reinforcement learning framework to facilitate the learning of contact-rich in-hand manipulation tasks, such as removing a lid. To this end, we propose tactile sensing-based reward shaping and incorporate the sensing into the observation space through embedding. The designed rewards encourage an agent to ensure firm grasping and smooth finger gaiting at the same time, leading to higher data efficiency and robust performance compared to the baseline. We verify the proposed framework on the opening a lid scenario, showing generalization of the trained policy into a couple of object types and various dynamics such as torsional friction. Lastly, the learned policy is demonstrated on the multi-fingered robot, Shadow Robot, showing that the control policy can be transferred to the real world. The video is available: https://youtu.be/poeJBPR7urQ.


Proprioceptive Image: An Image Representation of Proprioceptive Data from Quadruped Robots for Contact Estimation Learning

Abati, Gabriel Fischer, Soares, João Carlos Virgolino, Turrisi, Giulio, Barasuol, Victor, Semini, Claudio

arXiv.org Artificial Intelligence

Abstract-- This paper presents a novel approach for representing proprioceptive time-series data from quadruped robots as structured two-dimensional images, enabling the use of convolutional neural networks for learning locomotion-related tasks. The proposed method encodes temporal dynamics from multiple proprioceptive signals, such as joint positions, IMU readings, and foot velocities, while preserving the robot's morphological structure in the spatial arrangement of the image. We apply this concept in the problem of contact estimation, a key capability for stable and adaptive locomotion on diverse terrains. Experimental evaluations on both real-world datasets and simulated environments show that our image-based representation consistently enhances prediction accuracy and generalization over conventional sequence-based models, underscoring the potential of cross-modal encoding strategies for robotic state learning. Our method achieves superior performance on the contact dataset, improving contact state accuracy from 87.7% to 94.5% over the recently proposed MI-HGNN method, using a 15 times shorter window size. I. INTRODUCTION Deep learning has achieved remarkable success in domains where sequential or high-dimensional data can be transformed into visual representations suitable to convolu-tional architectures. In speech recognition, for instance, raw audio signals are often converted into spectrograms, two-dimensional time-frequency images, that serve as powerful inputs to convolutional neural networks (CNNs) and other image-based models.


Stand Up, NAO! Increasing the Reliability of Stand-Up Motions Through Error Compensation in Position Control

Reichenberg, Philip, Laue, Tim

arXiv.org Artificial Intelligence

Stand-up motions are an indispensable part of humanoid robot soccer. A robot incapable of standing up by itself is removed from the game for some time. In this paper, we present our stand-up motions for the NAO robot. Our approach dates back to 2019 and has been evaluated and slightly expanded over the past six years. We claim that the main reason for failed stand-up attempts are large errors in the executed joint positions. By addressing such problems by either executing special motions to free up stuck limbs such as the arms, or by compensating large errors with other joints, we significantly increased the overall success rate of our stand-up routine. The motions presented in this paper are also used by several other teams in the Standard Platform League, which thereby achieve similar success rates, as shown in an analysis of videos from multiple tournaments.


Dynamic Adaptive Legged Locomotion Policy via Decoupling Reaction Force Control and Gait Control

Wang, Renjie, Lyu, Shangke, Wang, Donglin

arXiv.org Artificial Intelligence

While Reinforcement Learning (RL) has achieved remarkable progress in legged locomotion control, it often suffers from performance degradation in out-of-distribution (OOD) conditions and discrepancies between the simulation and the real environments. Instead of mainly relying on domain randomization (DR) to best cover the real environments and thereby close the sim-to-real gap and enhance robustness, this work proposes an emerging decoupled framework that acquires fast online adaptation ability and mitigates the sim-to-real problems in unfamiliar environments by isolating stance-leg control and swing-leg control. Various simulation and real-world experiments demonstrate its effectiveness against horizontal force disturbances, uneven terrains, heavy and biased payloads, and sim-to-real gap.


Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Models

Ilic, Andela, Jiang, Jiaxi, Streli, Paul, Liu, Xintong, Holz, Christian

arXiv.org Artificial Intelligence

Motion capture using sparse inertial sensors has shown great promise due to its portability and lack of occlusion issues compared to camera-based tracking. Existing approaches typically assume that IMU sensors are tightly attached to the human body. However, this assumption often does not hold in real-world scenarios. In this paper, we present Garment Inertial Poser (GaIP), a method for estimating full-body poses from sparse and loosely attached IMU sensors. We first simulate IMU recordings using an existing garment-aware human motion dataset. Our transformer-based diffusion models synthesize loose IMU data and estimate human poses from this challenging loose IMU data. We also demonstrate that incorporating garment-related parameters during training on loose IMU data effectively maintains expressiveness and enhances the ability to capture variations introduced by looser or tighter garments. Our experiments show that our diffusion methods trained on simulated and synthetic data outperform state-of-the-art inertial full-body pose estimators, both quantitatively and qualitatively, opening up a promising direction for future research on motion capture from such realistic sensor placements.


EcBot: Data-Driven Energy Consumption Open-Source MATLAB Library for Manipulators

Heredia, Juan, Schlette, Christian, Kjærgaard, Mikkel Baun

arXiv.org Artificial Intelligence

Existing literature proposes models for estimating the electrical power of manipulators, yet two primary limitations prevail. First, most models are predominantly tested using traditional industrial robots. Second, these models often lack accuracy. To address these issues, we introduce an open source Matlab-based library designed to automatically generate \ac{ec} models for manipulators. The necessary inputs for the library are Denavit-Hartenberg parameters, link masses, and centers of mass. Additionally, our model is data-driven and requires real operational data, including joint positions, velocities, accelerations, electrical power, and corresponding timestamps. We validated our methodology by testing on four lightweight robots sourced from three distinct manufacturers: Universal Robots, Franka Emika, and Kinova. The model underwent testing, and the results demonstrated an RMSE ranging from 1.42 W to 2.80 W for the training dataset and from 1.45 W to 5.25 W for the testing dataset.


Reconfigurable legged metamachines that run on autonomous modular legs

Yu, Chen, Matthews, David, Wang, Jingxian, Gu, Jing, Blackiston, Douglas, Rubenstein, Michael, Kriegman, Sam

arXiv.org Artificial Intelligence

Legged machines are becoming increasingly agile and adaptive but they have so far lacked the morphological diversity of legged animals, which have been rearranged and reshaped to fill millions of niches. Unlike their biological counterparts, legged machines have largely converged over the past decade to canonical quadrupedal and bipedal architectures that cannot be easily reconfigured to meet new tasks or recover from injury. Here we introduce autonomous modular legs: agile yet minimal, single-degree-of-freedom jointed links that can learn complex dynamic behaviors and may be freely attached to form legged metamachines at the meter scale. This enables rapid repair, redesign, and recombination of highly-dynamic modular agents that move quickly and acrobatically (non-quasistatically) through unstructured environments. Because each module is itself a complete agent, legged metamachines are able to sustain deep structural damage that would completely disable other legged robots. We also show how to encode the vast space of possible body configurations into a compact latent design genome that can be efficiently explored, revealing a wide diversity of novel legged forms.


Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction

Tiezzi, Matteo, Apicella, Tommaso, Cardenas-Perez, Carlos, Fregonese, Giovanni, Dafarra, Stefano, Morerio, Pietro, Pucci, Daniele, Del Bue, Alessio

arXiv.org Artificial Intelligence

Evaluating and comparing the performance of autonomous Humanoid Robots is challenging, as success rate metrics are difficult to reproduce and fail to capture the complexity of robot movement trajectories, critical in Human-Robot Interaction and Collaboration (HRIC). To address these challenges, we propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance. We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories. NeME serves as a meta-evaluator to compare the performance of robot control policies, enabling policy evaluation without requiring human involvement in the loop. We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform. The experimental results indicate that our method is more aligned with the success rate obtained on the robot than baselines, offering a reproducible, systematic, and insightful means for comparing the performance of multimodal imitation learning approaches in complex HRI tasks.


ULC: A Unified and Fine-Grained Controller for Humanoid Loco-Manipulation

Sun, Wandong, Feng, Luying, Cao, Baoshi, Liu, Yang, Jin, Yaochu, Xie, Zongwu

arXiv.org Artificial Intelligence

--Loco-Manipulation for humanoid robots aims to enable robots to integrate mobility with upper-body tracking capabilities. Most existing approaches adopt hierarchical architectures that decompose control into isolated upper-body (manipulation) and lower-body (locomotion) policies. While this decomposition reduces training complexity, it inherently limits coordination between subsystems and contradicts the unified whole-body control exhibited by humans. We demonstrate that a single unified policy can achieve a combination of tracking accuracy, large workspace, and robustness for humanoid loco-manipulation. We propose the Unified Loco-Manipulation Controller (ULC), a single-policy framework that simultaneously tracks root velocity, root height, torso rotation, and dual-arm joint positions in an end-to-end manner, proving the feasibility of unified control without sacrificing performance. We achieve this unified control through key technologies: sequence skill acquisition for progressive learning complexity, residual action modeling for fine-grained control adjustments, command polynomial interpolation for smooth motion transitions, random delay release for robustness to deploy variations, load randomization for generalization to external disturbances, and center-of-gravity tracking for providing explicit policy gradients to maintain stability. Compared with strong baselines, ULC shows better tracking performance to disentangled methods and demonstrating larger workspace coverage. The unified dual-arm tracking enables precise manipulation under external loads while maintaining coordinated whole-body control for complex loco-manipulation tasks. The code and videos are available on our project website at https://ulc-humanoid.github.io. I. INTRODUCTION Humanoid robots, with their human-like morphology, represent a promising paradigm for versatile robotic systems capable of operating in human-designed environments.


PAPRLE (Plug-And-Play Robotic Limb Environment): A Modular Ecosystem for Robotic Limbs

Kwon, Obin, Yamsani, Sankalp, Myers, Noboru, Taylor, Sean, Hong, Jooyoung, Park, Kyungseo, Alspach, Alex, Kim, Joohyung

arXiv.org Artificial Intelligence

--We introduce PAPRLE (Plug-And-Play Robotic Limb Environment), a modular ecosystem that enables flexible placement and control of robotic limbs. With PAPRLE, a user can change the arrangement of the robotic limbs, and control them using a variety of input devices, including puppeteers, gaming controllers, and VR-based interfaces. This versatility supports a wide range of teleoperation scenarios and promotes adaptability to different task requirements. T o further enhance configurability, we introduce a pluggable puppeteer device that can be easily mounted and adapted to match the target robot configurations. PAPRLE supports bilateral teleoperation through these puppeteer devices, agnostic to the type or configuration of the follower robot. By supporting both joint-space and task-space control, the system provides real-time force feedback, improving user fidelity and physical interaction awareness. The modular design of PAPRLE facilitates novel spatial arrangements of the limbs and enables scalable data collection, thereby advancing research in embodied AI and learning-based control. The system will be released as open source, including both hardware and software components, to support broader adoption and community-driven extension. In many research and deployment scenarios, the flexible arrangement of robotic limbs is often required to support a wide range of task configurations.