This paper presents a shear-based control scheme for grasping and manipulating delicate objects with a Pisa/IIT anthropomorphic SoftHand equipped with soft biomimetic tactile sensors on all five fingertips. These `microTac' tactile sensors are miniature versions of the TacTip vision-based tactile sensor, and can extract precise contact geometry and force information at each fingertip for use as feedback into a controller to modulate the grasp while a held object is manipulated. Using a parallel processing pipeline, we asynchronously capture tactile images and predict contact pose and force from multiple tactile sensors. Consistent pose and force models across all sensors are developed using supervised deep learning with transfer learning techniques. We then develop a grasp control framework that uses contact force feedback from all fingertip sensors simultaneously, allowing the hand to safely handle delicate objects even under external disturbances. This control framework is applied to several grasp-manipulation experiments: first, retaining a flexible cup in a grasp without crushing it under changes in object weight; second, a pouring task where the center of mass of the cup changes dynamically; and third, a tactile-driven leader-follower task where a human guides a held object. These manipulation tasks demonstrate more human-like dexterity with underactuated robotic hands by using fast reflexive control from tactile sensing.

Abstract--This paper introduces a new generalized control method designed for multi-degrees-of-freedom devices to help people with limited motion capabilities in their daily activities. The challenge lies in finding the most adapted strategy for the control interface to effectively map user's motions in a lowdimensional space to complex robotic assistive devices, such as prostheses, supernumerary limbs, up to remote robotic avatars. The goal is a system which integrates the human and the robotic parts into a unique system, moving so as to reach the targets decided by the human while autonomously reducing the user's effort and discomfort. We present a framework to control general multi DoFs assistive systems, which translates user-performed compensatory motions into the necessary robot commands for reaching targets while canceling or reducing compensation. The framework extends to prostheses of any number of DoF up to full robotic avatars, regarded here as a sort of "whole-body prosthesis" of the person who sees the robot as an artificial extension of their own body without a physical link but with a sensory-motor integration. We have validated and applied this control strategy through tests encompassing simulated scenarios and real-world trials involving a virtual twin of the robotic parts (prosthesis and robot) and a physical humanoid avatar. SSISTIVE and rehabilitation devices such as powered wheelchairs, assistive robotic arms, and limb prostheses play a crucial role in assisting individuals with severe motor impairments [1], which require daily assistance due to e.g.

In recent years, artificial feet based on soft robotics and under-actuation principles emerged to improve mobility on challenging terrains. This paper presents the application of the MuJoCo physics engine to realize a digital twin of an adaptive soft foot developed for use with legged robots. We release the MuJoCo soft foot digital twin as open source to allow users and researchers to explore new approaches to locomotion. The work includes the system modeling techniques along with the kinematic and dynamic attributes involved. Validation is conducted through a rigorous comparison with bench tests on a physical prototype, replicating these experiments in simulation. Results are evaluated based on sole deformation and contact forces during foot-obstacle interaction. The foot model is subsequently integrated into simulations of the humanoid robot COMAN+, replacing its original flat feet. Results show an improvement in the robot's ability to negotiate small obstacles without altering its control strategy. Ultimately, this study offers a comprehensive modeling approach for adaptive soft feet, supported by qualitative comparisons of bipedal locomotion with state of the art robotic feet.

Intuitively, prostheses with user-controllable stiffness could mimic the intrinsic behavior of the human musculoskeletal system, promoting safe and natural interactions and task adaptability in real-world scenarios. However, prosthetic design often disregards compliance because of the additional complexity, weight, and needed control channels. This paper focuses on designing a Variable Stiffness Actuator (VSA) with weight, size, and performance compatible with prosthetic applications, addressing its implementation for the elbow joint. While a direct biomimetic approach suggests adopting an Agonist-Antagonist (AA) layout to replicate the biceps and triceps brachii with elastic actuation, this solution is not optimal to accommodate the varied morphologies of residual limbs. Instead, we employed the AA layout to craft an elbow prosthesis fully contained in the user's forearm, catering to individuals with distal transhumeral amputations. Additionally, we introduce a variant of this design where the two motors are split in the upper arm and forearm to distribute mass and volume more evenly along the bionic limb, enhancing comfort for patients with more proximal amputation levels. We characterize and validate our approach, demonstrating that both architectures meet the target requirements for an elbow prosthesis. The system attains the desired 120{\deg} range of motion, achieves the target stiffness range of [2, 60] Nm/rad, and can actively lift up to 3 kg. Our novel design reduces weight by up to 50% compared to existing VSAs for elbow prostheses while achieving performance comparable to the state of the art. Case studies suggest that passive and variable compliance could enable robust and safe interactions and task adaptability in the real world.

For tendon-driven multi-fingered robotic hands, ensuring grasp adaptability while minimizing the number of actuators needed to provide human-like functionality is a challenging problem. Inspired by the Pisa/IIT SoftHand, this paper introduces a 3D-printed, highly-underactuated, five-finger robotic hand named the Tactile SoftHand-A, which features only two actuators. The dual-tendon design allows for the active control of specific (distal or proximal interphalangeal) joints to adjust the hand's grasp gesture. We have also developed a new design of fully 3D-printed tactile sensor that requires no hand assembly and is printed directly as part of the robotic finger. This sensor is integrated into the fingertips and combined with the antagonistic tendon mechanism to develop a human-hand-guided tactile feedback grasping system. The system can actively mirror human hand gestures, adaptively stabilize grasp gestures upon contact, and adjust grasp gestures to prevent object movement after detecting slippage. Finally, we designed four different experiments to evaluate the novel fingers coupled with the antagonistic mechanism for controlling the robotic hand's gestures, adaptive grasping ability, and human-hand-guided tactile feedback grasping capability. The experimental results demonstrate that the Tactile SoftHand-A can adaptively grasp objects of a wide range of shapes and automatically adjust its gripping gestures upon detecting contact and slippage. Overall, this study points the way towards a class of low-cost, accessible, 3D-printable, underactuated human-like robotic hands, and we openly release the designs to facilitate others to build upon this work. This work is Open-sourced at github.com/SoutheastWind/Tactile_SoftHand_A

Our research investigates vibrotactile perception in four prosthetic hands with distinct kinematics and mechanical characteristics. We found that rigid and simple socket-based prosthetic devices can transmit tactile information and surprisingly enable users to identify the stimulated finger with high reliability. This ability decreases with more advanced prosthetic hands with additional articulations and softer mechanics. We conducted experiments to understand the underlying mechanisms. We assessed a prosthetic user's ability to discriminate finger contacts based on vibrations transmitted through the four prosthetic hands. We also performed numerical and mechanical vibration tests on the prostheses and used a machine learning classifier to identify the contacted finger. Our results show that simpler and rigid prosthetic hands facilitate contact discrimination (for instance, a user of a purely cosmetic hand can distinguish a contact on the index finger from other fingers with 83% accuracy), but all tested hands, including soft advanced ones, performed above chance level. Despite advanced hands reducing vibration transmission, a machine learning algorithm still exceeded human performance in discriminating finger contacts. These findings suggest the potential for enhancing vibrotactile feedback in advanced prosthetic hands and lay the groundwork for future integration of such feedback in prosthetic devices.

Biped robots usually adopt feet with a rigid structure that simplifies walking on flat grounds and yet hinders ground adaptation in unstructured environments, thus jeopardizing stability. We recently explored in the SoftFoot the idea of adapting a robotic foot to ground irregularities along the sagittal plane. Building on the previous results, we propose in this paper a novel robotic foot able to adapt both in the sagittal and frontal planes, similarly to the human foot. It features five parallel modules with intrinsic longitudinal adaptability that can be combined in many possible designs through optional rigid or elastic connections. By following a methodological design approach, we narrow down the design space to five candidate foot designs and implement them on a modular system. Prototypes are tested experimentally via controlled application of force, through a robotic arm, onto a sensorized plate endowed with different obstacles. Their performance is compared, using also a rigid foot and the previous SoftFoot as a baseline. Analysis of footprint stability shows that the introduction of the transverse arch, by elastically connecting the five parallel modules, is advantageous for obstacle negotiation, especially when obstacles are located under the forefoot. In addition to biped robots' locomotion, this finding might also benefit lower-limb prostheses design.

This paper presents an aerial platform capable of performing physically interactive tasks in unstructured environments with human-like dexterity under human supervision. This aerial platform consists of a humanoid torso attached to a hexacopter. A two-degree-of-freedom head and two five-degree-of-freedom arms equipped with softhands provide the requisite dexterity to allow human operators to carry out various tasks. A robust tendon-driven structure is purposefully designed for the arms, considerably reducing the impact of arm inertia on the floating base in motion. In addition, tendons provide flexibility to the joints, which enhances the robustness of the arm preventing damage in interaction with the environment. To increase the payload of the aerial system and the battery life, we use the concept of Suspended Aerial Manipulation, i.e., the flying humanoid can be connected with a tether to a structure, e.g., a larger airborne carrier or a supporting crane. Importantly, to maximize portability and applicability, we adopt a modular approach exploiting commercial components for the aerial base hardware and autopilot, while developing an outer stabilizing control loop to maintain the attitude, compensating for the tether force and for the humanoid head and arm motions. The humanoid can be controlled by a remote operator, thus effectively realizing a Suspended Aerial Manipulation Avatar. The proposed system is validated through experiments in indoor scenarios reproducing post-disaster tasks.

Robot feet are crucial for maintaining dynamic stability and propelling the body during walking, especially on uneven terrains. Traditionally, robot feet were mostly designed as flat and stiff pieces of metal, which meets its limitations when the robot is required to step on irregular grounds, e.g. stones. While one could think that adding compliance under such feet would solve the problem, this is not the case. To address this problem, we introduced the SoftFoot, an adaptive foot design that can enhance walking performance over irregular grounds. The proposed design is completely passive and varies its shape and stiffness based on the exerted forces, through a system of pulley, tendons, and springs opportunely placed in the structure. This paper outlines the motivation behind the SoftFoot and describes the theoretical model which led to its final design. The proposed system has been experimentally tested and compared with two analogous conventional feet, a rigid one and a compliant one, with similar footprints and soles. The experimental validation focuses on the analysis of the standing performance, measured in terms of the equivalent support surface extension and the compensatory ankle angle, and the rejection of impulsive forces, which is important in events such as stepping on unforeseen obstacles. Results show that the SoftFoot has the largest equivalent support surface when standing on obstacles, and absorbs impulsive loads in a way almost as good as a compliant foot.

The ANA Avatar XPRIZE was a four-year competition to develop a robotic "avatar" system to allow a human operator to sense, communicate, and act in a remote environment as though physically present. The competition featured a unique requirement that judges would operate the avatars after less than one hour of training on the human-machine interfaces, and avatar systems were judged on both objective and subjective scoring metrics. This paper presents a unified summary and analysis of the competition from technical, judging, and organizational perspectives. We study the use of telerobotics technologies and innovations pursued by the competing teams in their avatar systems, and correlate the use of these technologies with judges' task performance and subjective survey ratings. It also summarizes perspectives from team leads, judges, and organizers about the competition's execution and impact to inform the future development of telerobotics and telepresence.