della santina
Bridging Embodiment Gaps: Deploying Vision-Language-Action Models on Soft Robots
Su, Haochen, Meo, Cristian, Stella, Francesco, Peirone, Andrea, Junge, Kai, Hughes, Josie
Robotic systems are increasingly expected to operate in human-centered, unstructured environments where safety, adaptability, and generalization are essential. Vision-Language-Action (VLA) models have been proposed as a language guided generalized control framework for real robots. However, their deployment has been limited to conventional serial link manipulators. Coupled by their rigidity and unpredictability of learning based control, the ability to safely interact with the environment is missing yet critical. In this work, we present the deployment of a VLA model on a soft continuum manipulator to demonstrate autonomous safe human-robot interaction. We present a structured finetuning and deployment pipeline evaluating two state-of-the-art VLA models (OpenVLA-OFT and $π_0$) across representative manipulation tasks, and show while out-of-the-box policies fail due to embodiment mismatch, through targeted finetuning the soft robot performs equally to the rigid counterpart. Our findings highlight the necessity of finetuning for bridging embodiment gaps, and demonstrate that coupling VLA models with soft robots enables safe and flexible embodied AI in human-shared environments.
- Europe > Switzerland > Vaud > Lausanne (0.05)
- Europe > Netherlands > South Holland > Delft (0.04)
- Asia > Japan > Honshū > Chūbu > Toyama Prefecture > Toyama (0.04)
- Asia > Japan > Honshū > Chūbu > Ishikawa Prefecture > Kanazawa (0.04)
Soft yet Effective Robots via Holistic Co-Design
Stölzle, Maximilian, Pagliarani, Niccolò, Stella, Francesco, Hughes, Josie, Laschi, Cecilia, Rus, Daniela, Cianchetti, Matteo, Della Santina, Cosimo, Zardini, Gioele
Soft robots promise inherent safety via their material compliance for seamless interactions with humans or delicate environments. Yet, their development is challenging because it requires integrating materials, geometry, actuation, and autonomy into complex mechatronic systems. Despite progress, the field struggles to balance task-specific performance with broader factors like durability and manufacturability - a difficulty that we find is compounded by traditional sequential design processes with their lack of feedback loops. In this perspective, we review emerging co-design approaches that simultaneously optimize the body and brain, enabling the discovery of unconventional designs highly tailored to the given tasks. We then identify three key shortcomings that limit the broader adoption of such co-design methods within the soft robotics domain. First, many rely on simulation-based evaluations focusing on a single metric, while real-world designs must satisfy diverse criteria. Second, current methods emphasize computational modeling without ensuring feasible realization, risking sim-to-real performance gaps. Third, high computational demands limit the exploration of the complete design space. Finally, we propose a holistic co-design framework that addresses these challenges by incorporating a broader range of design values, integrating real-world prototyping to refine evaluations, and boosting efficiency through surrogate metrics and model-based control strategies. This holistic framework, by simultaneously optimizing functionality, durability, and manufacturability, has the potential to enhance reliability and foster broader acceptance of soft robotics, transforming human-robot interactions.
- Europe > Netherlands > South Holland > Delft (0.04)
- Asia > Singapore (0.04)
- Europe > Italy (0.04)
- (7 more...)
- Research Report (1.00)
- Overview (1.00)
Explosive Jumping with Rigid and Articulated Soft Quadrupeds via Example Guided Reinforcement Learning
Apostolides, Georgios, Pan, Wei, Kober, Jens, Della Santina, Cosimo, Ding, Jiatao
Achieving controlled jumping behaviour for a quadruped robot is a challenging task, especially when introducing passive compliance in mechanical design. This study addresses this challenge via imitation-based deep reinforcement learning with a progressive training process. To start, we learn the jumping skill by mimicking a coarse jumping example generated by model-based trajectory optimization. Subsequently, we generalize the learned policy to broader situations, including various distances in both forward and lateral directions, and then pursue robust jumping in unknown ground unevenness. In addition, without tuning the reward much, we learn the jumping policy for a quadruped with parallel elasticity. Results show that using the proposed method, i) the robot learns versatile jumps by learning only from a single demonstration, ii) the robot with parallel compliance reduces the landing error by 11.1%, saves energy cost by 15.2% and reduces the peak torque by 15.8%, compared to the rigid robot without parallel elasticity, iii) the robot can perform jumps of variable distances with robustness against ground unevenness (maximal 4cm height perturbations) using only proprioceptive perception.
- Europe > Netherlands > South Holland > Delft (0.04)
- Europe > United Kingdom > England > Greater Manchester > Manchester (0.04)
- Europe > Germany (0.04)
Clarke Coordinates Are Generalized Improved State Parametrization for Continuum Robots
Grassmann, Reinhard M., Burgner-Kahrs, Jessica
In this letter, we demonstrate that previously proposed improved state parameterizations for soft and continuum robots are specific cases of Clarke coordinates. By explicitly deriving these improved parameterizations from a generalized Clarke transformation matrix, we unify various approaches into one comprehensive mathematical framework. This unified representation provides clarity regarding their relationships and generalizes them beyond existing constraints, including arbitrary joint numbers, joint distributions, and underlying modeling assumptions. This unification consolidates prior insights and establishes Clarke coordinates as a foundational tool, enabling systematic knowledge transfer across different subfields within soft and continuum robotics.
- North America > Canada > Ontario > Toronto (0.14)
- Asia (0.04)
Design and Control of Modular Soft-Rigid Hybrid Manipulators with Self-Contact
Patterson, Zach J., Sologuren, Emily, Della Santina, Cosimo, Rus, Daniela
Soft robotics focuses on designing robots with highly deformable materials, allowing them to adapt and operate safely and reliably in unstructured and variable environments. While soft robots offer increased compliance over rigid body robots, their payloads are limited, and they consume significant energy when operating against gravity in terrestrial environments. To address the carrying capacity limitation, we introduce a novel class of soft-rigid hybrid robot manipulators (SRH) that incorporates both soft continuum modules and rigid joints in a serial configuration. The SRH manipulators can seamlessly transition between being compliant and delicate to rigid and strong, achieving this through dynamic shape modulation and employing self-contact among rigid components to effectively form solid structures. We discuss the design and fabrication of SRH robots, and present a class of novel control algorithms for SRH systems. We propose a configuration space PD+ shape controller and a Cartesian impedance controller, both of which are provably stable, endowing the soft robot with the necessary low-level capabilities. We validate the controllers on SRH hardware and demonstrate the robot performing several tasks. Our results highlight the potential for the soft-rigid hybrid paradigm to produce robots that are both physically safe and effective at task performance.
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.14)
- Europe > United Kingdom > England > Greater London > London (0.04)
- Europe > Netherlands > South Holland > Delft (0.04)
- (3 more...)
Berry Twist: a Twisting-Tube Soft Robotic Gripper for Blackberry Harvesting
Elfferich, Johannes F., Shahabi, Ebrahim, Della Santina, Cosimo, Dodou, Dimitra
As global demand for fruits and vegetables continues to rise, the agricultural industry faces challenges in securing adequate labor. Robotic harvesting devices offer a promising solution to solve this issue. However, harvesting delicate fruits, notably blackberries, poses unique challenges due to their fragility. This study introduces and evaluates a prototype robotic gripper specifically designed for blackberry harvesting. The gripper features an innovative fabric tube mechanism employing motorized twisting action to gently envelop the fruit, ensuring uniform pressure application and minimizing damage. Three types of tubes were developed, varying in elasticity and compressibility using foam padding, spandex, and food-safe cotton cheesecloth. Performance testing focused on assessing each gripper's ability to detach and release blackberries, with emphasis on quantifying damage rates. Results indicate the proposed gripper achieved an 82% success rate in detaching blackberries and a 95% success rate in releasing them, showcasing the promised potential for robotic harvesting applications.
Awareness in robotics: An early perspective from the viewpoint of the EIC Pathfinder Challenge "Awareness Inside''
Della Santina, Cosimo, Corbato, Carlos Hernandez, Sisman, Burak, Leiva, Luis A., Arapakis, Ioannis, Vakalellis, Michalis, Vanderdonckt, Jean, D'Haro, Luis Fernando, Manzi, Guido, Becchio, Cristina, Elamrani, Aïda, Alirezaei, Mohsen, Castellano, Ginevra, Dimarogonas, Dimos V., Ghosh, Arabinda, Haesaert, Sofie, Soudjani, Sadegh, Stroeve, Sybert, Verschure, Paul, Bacciu, Davide, Deroy, Ophelia, Bahrami, Bahador, Gallicchio, Claudio, Hauert, Sabine, Sanz, Ricardo, Lanillos, Pablo, Iacca, Giovanni, Sigg, Stephan, Gasulla, Manel, Steels, Luc, Sierra, Carles
Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars in the past. However, things are changing as more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices that are connected with projects funded by the EIC Pathfinder Challenge called "Awareness Inside", a nonrecurring call for proposals within Horizon Europe that was designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems.
- Europe > Spain > Galicia > Madrid (0.05)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Europe > Sweden > Uppsala County > Uppsala (0.04)
- (16 more...)
- Government (0.95)
- Health & Medicine > Therapeutic Area > Neurology (0.69)
An Experimental Study of Model-based Control for Planar Handed Shearing Auxetics Robots
Stölzle, Maximilian, Rus, Daniela, Della Santina, Cosimo
Parallel robots based on Handed Shearing Auxetics (HSAs) can implement complex motions using standard electric motors while maintaining the complete softness of the structure, thanks to specifically designed architected metamaterials. However, their control is especially challenging due to varying and coupled stiffness, shearing, non-affine terms in the actuation model, and underactuation. In this paper, we present a model-based control strategy for planar HSA robots enabling regulation in task space. We formulate equations of motion, show that they admit a collocated form, and design a P-satI-D feedback controller with compensation for elastic and gravitational forces. We experimentally identify and verify the proposed control strategy in closed loop.
- Europe > Netherlands > South Holland > Delft (0.05)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- Research Report > Experimental Study (0.50)
- Research Report > New Finding (0.40)
Guiding Soft Robots with Motor-Imagery Brain Signals and Impedance Control
Stölzle, Maximilian, Baberwal, Sonal Santosh, Rus, Daniela, Coyle, Shirley, Della Santina, Cosimo
Integrating Brain-Machine Interfaces into non-clinical applications like robot motion control remains difficult - despite remarkable advancements in clinical settings. Specifically, EEG-based motor imagery systems are still error-prone, posing safety risks when rigid robots operate near humans. This work presents an alternative pathway towards safe and effective operation by combining wearable EEG with physically embodied safety in soft robots. We introduce and test a pipeline that allows a user to move a soft robot's end effector in real time via brain waves that are measured by as few as three EEG channels. A robust motor imagery algorithm interprets the user's intentions to move the position of a virtual attractor to which the end effector is attracted, thanks to a new Cartesian impedance controller. We specifically focus here on planar soft robot-based architected metamaterials, which require the development of a novel control architecture to deal with the peculiar nonlinearities - e.g., non-affinity in control. We preliminarily but quantitatively evaluate the approach on the task of setpoint regulation. We observe that the user reaches the proximity of the setpoint in 66% of steps and that for successful steps, the average response time is 21.5s. We also demonstrate the execution of simple real-world tasks involving interaction with the environment, which would be extremely hard to perform if it were not for the robot's softness.
- Europe > Netherlands > South Holland > Delft (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- Europe > Ireland > Leinster > County Dublin > Dublin (0.04)
- Europe > Austria > Styria > Graz (0.04)
Modeling and Control of Intrinsically Elasticity Coupled Soft-Rigid Robots
Patterson, Zach J., Della Santina, Cosimo, Rus, Daniela
While much work has been done recently in the realm of model-based control of soft robots and soft-rigid hybrids, most works examine robots that have an inherently serial structure. While these systems have been prevalent in the literature, there is an increasing trend toward designing soft-rigid hybrids with intrinsically coupled elasticity between various degrees of freedom. In this work, we seek to address the issues of modeling and controlling such structures, particularly when underactuated. We introduce several simple models for elastic coupling, typical of those seen in these systems. We then propose a controller that compensates for the elasticity, and we prove its stability with Lyapunov methods without relying on the elastic dominance assumption. This controller is applicable to the general class of underactuated soft robots. After evaluating the controller in simulated cases, we then develop a simple hardware platform to evaluate both the models and the controller. Finally, using the hardware, we demonstrate a novel use case for underactuated, elastically coupled systems in "sensorless" force control.
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- North America > United States > Michigan (0.04)
- Europe > Netherlands > South Holland > Delft (0.04)