Goto

Collaborating Authors

 robotic simulator


A Preview of HoloOcean 2.0

arXiv.org Artificial Intelligence

Abstract-- Marine robotics simulators play a fundamental role in the development of marine robotic systems. With increased focus on the marine robotics field in recent years, there has been significant interest in developing higher fidelity simulation of marine sensors, physics, and visual rendering capabilities to support autonomous marine robot development and validation. HoloOcean 2.0, the next major release of HoloOcean, brings state-of-the-art features under a general marine simulator capable of supporting a variety of tasks. New features in HoloOcean 2.0 include migration to Unreal Engine (UE) 5.3, advanced vehicle dynamics using models from Fossen, and support for ROS2 using a custom bridge. Additional features are currently in development, including significantly more efficient ray tracing-based sidescan, forward-looking, and bathymetric sonar implementations; semantic sensors; environment generation tools; volumetric environmental effects; and realistic waves. Marine robotics simulators have supported research and development for autonomous underwater and surface vessels for several decades.


HuNavSim 2.0: An Enhanced Human Navigation Simulator for Human-Aware Robot Navigation

arXiv.org Artificial Intelligence

This work presents a new iteration of the Human Navigation Simulator (HuNavSim), a novel open-source tool for the simulation of different human-agent navigation behaviors in scenarios with mobile robots. The tool, programmed under the ROS 2 framework, can be used together with different well-known robotics simulators such as Gazebo or NVidia Isaac Sim. The main goal is to facilitate the development and evaluation of human-aware robot navigation systems in simulation. In this new version, several features have been improved and new ones added, such as the extended set of actions and conditions that can be combined in Behavior Trees to compound complex and realistic human behaviors.


MEbots: Integrating a RISC-V Virtual Platform with a Robotic Simulator for Energy-aware Design

arXiv.org Artificial Intelligence

Personal use of this material is permitted. Abstract --Virtual Platforms (VPs) enable early software validation of autonomous systems' electronics, reducing costs and time-to-market. While many VPs support both functional and non-functional simulation (e.g., timing, power), they lack the capability of simulating the environment in which the system operates. In contrast, robotics simulators lack accurate timing and power features. This twofold shortcoming limits the effectiveness of the design flow, as the designer can not fully evaluate the features of the solution under development. This paper presents a novel, fully open-source framework bridging this gap by integrating a robotics simulator (Webots) with a VP for RISC-V-based systems (MESSY). The framework enables a holistic, mission-level, energy-aware co-simulation of electronics in their surrounding environment, streamlining the exploration of design configurations and advanced power management policies. Virtual Platforms (VPs) enable comprehensive system modeling and simulation before physical production [1] and are thus a crucial resource in the design of modern embedded systems, characterized by heterogeneity and tight integration with the physical environment.


MARVIS: Motion & Geometry Aware Real and Virtual Image Segmentation

arXiv.org Artificial Intelligence

Tasks such as autonomous navigation, 3D reconstruction, and object recognition near the water surfaces are crucial in marine robotics applications. However, challenges arise due to dynamic disturbances, e.g., light reflections and refraction from the random air-water interface, irregular liquid flow, and similar factors, which can lead to potential failures in perception and navigation systems. Traditional computer vision algorithms struggle to differentiate between real and virtual image regions, significantly complicating tasks. A virtual image region is an apparent representation formed by the redirection of light rays, typically through reflection or refraction, creating the illusion of an object's presence without its actual physical location. This work proposes a novel approach for segmentation on real and virtual image regions, exploiting synthetic images combined with domain-invariant information, a Motion Entropy Kernel, and Epipolar Geometric Consistency. Our segmentation network does not need to be re-trained if the domain changes. We show this by deploying the same segmentation network in two different domains: simulation and the real world. By creating realistic synthetic images that mimic the complexities of the water surface, we provide fine-grained training data for our network (MARVIS) to discern between real and virtual images effectively. By motion & geometry-aware design choices and through comprehensive experimental analysis, we achieve state-of-the-art real-virtual image segmentation performance in unseen real world domain, achieving an IoU over 78% and a F1-Score over 86% while ensuring a small computational footprint. MARVIS offers over 43 FPS (8 FPS) inference rates on a single GPU (CPU core). Our code and dataset are available here https://github.com/jiayi-wu-umd/MARVIS.


Developing Simulation Models for Soft Robotic Grippers in Webots

arXiv.org Artificial Intelligence

Robotic simulators provide cost-effective and risk-free virtual environments for studying robotic designs, control algorithms, and sensor integrations. They typically host extensive libraries of sensors and actuators that facilitate rapid prototyping and design evaluations in simulation. The use of the most prominent existing robotic simulators is however limited to simulation of rigid-link robots. On the other hand, there exist dedicated specialized environments for simulating soft robots. This separation limits the study of soft robotic systems, particularly in hybrid scenarios where soft and rigid sub-systems co-exist. In this work, we develop a lightweight open-source digital twin of a commercially available soft gripper, directly integrated within the robotic simulator Webots. We use a Rigid-Link-Discretization (RLD) model to simulate the soft gripper. Using a Particle Swarm Optimization (PSO) approach, we identify the parameters of the RLD model based on the kinematics and dynamics of the physical system and show the efficacy of our modeling approach in validation experiments. All software and experimental details are available on github: https://github.com/anonymousgituser1/Robosoft2025


SliceIt! -- A Dual Simulator Framework for Learning Robot Food Slicing

arXiv.org Artificial Intelligence

Cooking robots can enhance the home experience by reducing the burden of daily chores. However, these robots must perform their tasks dexterously and safely in shared human environments, especially when handling dangerous tools such as kitchen knives. This study focuses on enabling a robot to autonomously and safely learn food-cutting tasks. More specifically, our goal is to enable a collaborative robot or industrial robot arm to perform food-slicing tasks by adapting to varying material properties using compliance control. Our approach involves using Reinforcement Learning (RL) to train a robot to compliantly manipulate a knife, by reducing the contact forces exerted by the food items and by the cutting board. However, training the robot in the real world can be inefficient, and dangerous, and result in a lot of food waste. Therefore, we proposed SliceIt!, a framework for safely and efficiently learning robot food-slicing tasks in simulation. Following a real2sim2real approach, our framework consists of collecting a few real food slicing data, calibrating our dual simulation environment (a high-fidelity cutting simulator and a robotic simulator), learning compliant control policies on the calibrated simulation environment, and finally, deploying the policies on the real robot.


Robotics as a Simulation Educational Tool

arXiv.org Artificial Intelligence

In the evolving landscape of education, robotics has emerged as a powerful tool for fostering creativity, critical thinking, and problem-solving skills among students of all ages. This innovative approach to learning seamlessly integrates STEM (Science, Technology, Engineering, and Mathematics) concepts, creating an engaging and immersive learning experience. Educational robotics transcends traditional classroom settings, transforming learning into a hands-on, experiential endeavor. Students are actively involved in the design, construction, and programming of robots, allowing them to apply theoretical concepts to practical applications. This hands-on approach fosters deeper understanding and retention of knowledge, making learning more meaningful and enjoyable. In this paper, the potential of simulation robotics is evaluated as a hands on interactive learning experience that goes beyond traditional robotic classroom methods.


Potato: A Data-Oriented Programming 3D Simulator for Large-Scale Heterogeneous Swarm Robotics

arXiv.org Artificial Intelligence

Large-scale simulation with realistic nonlinear dynamic models is crucial for algorithms development for swarm robotics. However, existing platforms are mainly developed based on Object-Oriented Programming (OOP) and either use simple kinematic models to pursue a large number of simulating nodes or implement realistic dynamic models with limited simulating nodes. In this paper, we develop a simulator based on Data-Oriented Programming (DOP) that utilizes GPU parallel computing to achieve large-scale swarm robotic simulations. Specifically, we use a multi-process approach to simulate heterogeneous agents and leverage PyTorch with GPU to simulate homogeneous agents with a large number. We test our approach using a nonlinear quadrotor model and demonstrate that this DOP approach can maintain almost the same computational speed when quadrotors are less than 5,000. We also provide two examples to present the functionality of the platform.


Simulations for mobile robots

#artificialintelligence

What I like the most about robotic simulations is their sheer ability to make software development and testing process time-efficient. Working with robots (to a large extent on prototypes, and often remotely) over the last decade has helped me come up with a simple rule -- do as much as you can with the simulation, use the actual robot hardware when you absolutely have to. Software for robots HAS TO run on robots, there is no way around it. However, there is plenty of simulation-based testing that can expedite your route to software deployment on the robot, and robot deployment on-site. I've spent the bulk of my time working with wheeled mobile robots and my choice of simulators for application development and testing is centered around that.