Goto

Collaborating Authors

 vial


PREVENT: Proactive Risk Evaluation and Vigilant Execution of Tasks for Mobile Robotic Chemists using Multi-Modal Behavior Trees

Veeramani, Satheeshkumar, Zhou, Zhengxue, Munguia-Galeano, Francisco, Fakhruldeen, Hatem, Roddelkopf, Thomas, Al-Okby, Mohammed Faeik Ruzaij, Thurow, Kerstin, Cooper, Andrew Ian

arXiv.org Artificial Intelligence

Mobile robotic chemists are a fast growing trend in the field of chemistry and materials research. However, so far these mobile robots lack workflow awareness skills. This poses the risk that even a small anomaly, such as an improperly capped sample vial could disrupt the entire workflow. This wastes time, and resources, and could pose risks to human researchers, such as exposure to toxic materials. Existing perception mechanisms can be used to predict anomalies but they often generate excessive false positives. This may halt workflow execution unnecessarily, requiring researchers to intervene and to resume the workflow when no problem actually exists, negating the benefits of autonomous operation. To address this problem, we propose PREVENT a system comprising navigation and manipulation skills based on a multimodal Behavior Tree (BT) approach that can be integrated into existing software architectures with minimal modifications. Our approach involves a hierarchical perception mechanism that exploits AI techniques and sensory feedback through Dexterous Vision and Navigational Vision cameras and an IoT gas sensor module for execution-related decision-making. Experimental evaluations show that the proposed approach is comparatively efficient and completely avoids both false negatives and false positives when tested in simulated risk scenarios within our robotic chemistry workflow. The results also show that the proposed multi-modal perception skills achieved deployment accuracies that were higher than the average of the corresponding uni-modal skills, both for navigation and for manipulation.


A Appendix A.1 Dataset An example of a single state from a nullS t,A,S

Neural Information Processing Systems

Game: ztuu Location: Cultural Complex This imposing ante-room, the center of what was apparently the cultural center of the GUE, is adorned in the ghastly style of the GUE's "Grotesque Period." With leering gargoyles, cartoonish friezes depicting long-forgotten scenes of GUE history, and primitive statuary of pointy-headed personages unknown (perhaps very, very distant progenitors of the Flatheads), the place would have been best left undiscovered. North of here, a large hallway passes under the roughly hewn inscription "Convention Center." To the east, under a fifty-story triumphal arch, a passageway the size of a large city boulevard opens into the Royal Theater. A relatively small and unobtrusive sign (perhaps ten feet high) stands nearby.


AutoLabs: Cognitive Multi-Agent Systems with Self-Correction for Autonomous Chemical Experimentation

Panapitiya, Gihan, Saldanha, Emily, Job, Heather, Hess, Olivia

arXiv.org Artificial Intelligence

The automation of chemical research through self-driving laboratories (SDLs) promises to accelerate scientific discovery, yet the reliability and granular performance of the underlying AI agents remain critical, under-examined challenges. In this work, we introduce AutoLabs, a self-correcting, multi-agent architecture designed to autonomously translate natural-language instructions into executable protocols for a high-throughput liquid handler. The system engages users in dialogue, decomposes experimental goals into discrete tasks for specialized agents, performs tool-assisted stoichiometric calculations, and iteratively self-corrects its output before generating a hardware-ready file. We present a comprehensive evaluation framework featuring five benchmark experiments of increasing complexity, from simple sample preparation to multi-plate timed syntheses. Through a systematic ablation study of 20 agent configurations, we assess the impact of reasoning capacity, architectural design (single- vs. multi-agent), tool use, and self-correction mechanisms. Our results demonstrate that agent reasoning capacity is the most critical factor for success, reducing quantitative errors in chemical amounts (nRMSE) by over 85% in complex tasks. When combined with a multi-agent architecture and iterative self-correction, AutoLabs achieves near-expert procedural accuracy (F1-score > 0.89) on challenging multi-step syntheses. These findings establish a clear blueprint for developing robust and trustworthy AI partners for autonomous laboratories, highlighting the synergistic effects of modular design, advanced reasoning, and self-correction to ensure both performance and reliability in high-stakes scientific applications. Code: https://github.com/pnnl/autolabs


ARChemist: Autonomous Robotic Chemistry System Architecture

Fakhruldeen, Hatem, Pizzuto, Gabriella, Glowacki, Jakub, Cooper, Andrew Ian

arXiv.org Artificial Intelligence

-- Automated laboratory experiments have the potential to propel new discoveries, while increasing reproducibility and improving scientists' safety when handling dangerous materials. However, many automated laboratory workflows have not fully leveraged the remarkable advancements in robotics and digital lab equipment. As a result, most robotic systems used in the labs are programmed specifically for a single experiment, often relying on proprietary architectures or using unconventional hardware. In this work, we tackle this problem by proposing a novel robotic system architecture specifically designed with and for chemists, which allows the scientist to easily reconfigure their setup for new experiments. Specifically, the system's strength is its ability to combine together heterogeneous robotic platforms with standard laboratory equipment to create different experimental setups. Finally, we show how the architecture can be used for specific laboratory experiments through case studies such as solubility screening and crystallisation. I. INTRODUCTION Accelerating the discovery of new materials is important for industrial applications such as healthcare and energy production. This can be achieved through running long-term experiments autonomously, for example by increasing the use of robotic platforms in laboratories. In practice, this would accumulate more experiments in less time, and potentially minimise the scientists' exposure to harmful chemicals, reducing their repetitive tasks.


Multimodal Behaviour Trees for Robotic Laboratory Task Automation

Fakhruldeen, Hatem, Nambiar, Arvind Raveendran, Veeramani, Satheeshkumar, Tailor, Bonilkumar Vijaykumar, Juneghani, Hadi Beyzaee, Pizzuto, Gabriella, Cooper, Andrew Ian

arXiv.org Artificial Intelligence

Laboratory robotics offer the capability to conduct experiments with a high degree of precision and reproducibility, with the potential to transform scientific research. Trivial and repeatable tasks; e.g., sample transportation for analysis and vial capping are well-suited for robots; if done successfully and reliably, chemists could contribute their efforts towards more critical research activities. Currently, robots can perform these tasks faster than chemists, but how reliable are they? Improper capping could result in human exposure to toxic chemicals which could be fatal. To ensure that robots perform these tasks as accurately as humans, sensory feedback is required to assess the progress of task execution. To address this, we propose a novel methodology based on behaviour trees with multimodal perception. Along with automating robotic tasks, this methodology also verifies the successful execution of the task, a fundamental requirement in safety-critical environments. The experimental evaluation was conducted on two lab tasks: sample vial capping and laboratory rack insertion. The results show high success rate, i.e., 88% for capping and 92% for insertion, along with strong error detection capabilities. This ultimately proves the robustness and reliability of our approach and that using multimodal behaviour trees should pave the way towards the next generation of robotic chemists.


An Open-source Capping Machine Suitable for Confined Spaces

Munguia-Galeano, Francisco, Longley, Louis, Veeramani, Satheeshkumar, Zhou, Zhengxue, Clowes, Rob, Fakhruldeen, Hatem, Cooper, Andrew I.

arXiv.org Artificial Intelligence

In the context of self-driving laboratories (SDLs), ensuring automated and error-free capping is crucial, as it is a ubiquitous step in sample preparation. Automated capping in SDLs can occur in both large and small workspaces (e.g., inside a fume hood). However, most commercial capping machines are designed primarily for large spaces and are often too bulky for confined environments. Moreover, many commercial products are closed-source, which can make their integration into fully autonomous workflows difficult. This paper introduces an open-source capping machine suitable for compact spaces, which also integrates a vision system that recognises capping failure. The capping and uncapping processes are repeated 100 times each to validate the machine's design and performance. As a result, the capping machine reached a 100 % success rate for capping and uncapping. Furthermore, the machine sealing capacities are evaluated by capping 12 vials filled with solvents of different vapour pressures: water, ethanol and acetone. The vials are then weighed every 3 hours for three days. The machine's performance is benchmarked against an industrial capping machine (a Chemspeed station) and manual capping. The vials capped with the prototype lost 0.54 % of their content weight on average per day, while the ones capped with the Chemspeed and manually lost 0.0078 % and 0.013 %, respectively. The results show that the capping machine is a reasonable alternative to industrial and manual capping, especially when space and budget are limitations in SDLs.


Could We Store Our Data in DNA?

The New Yorker

A zettabyte is a trillion gigabytes. That's a lot--but, according to one estimate, humanity will produce a hundred and eighty zettabytes of digital data this year. It all adds up: PowerPoints and selfies; video captured by cameras; electronic health records; data retrieved from smart devices or collected by telescopes and particle accelerators; backups, and backups of the backups. Where should it all go, and how much of it should be kept, and for how long? These questions vex the computer scientists who manage the world's storage. For them, the cloud isn't nebulous but a physical system that must be built, paid for, and maintained.


Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Models

Turcato, Niccolò, Iovino, Matteo, Synodinos, Aris, Libera, Alberto Dalla, Carli, Ruggero, Falco, Pietro

arXiv.org Artificial Intelligence

Recent advancements in Large Language Models (LLMs) and Visual Language Models (VLMs) have significantly impacted robotics, enabling high-level semantic motion planning applications. Reinforcement Learning (RL), a complementary paradigm, enables agents to autonomously optimize complex behaviors through interaction and reward signals. However, designing effective reward functions for RL remains challenging, especially in real-world tasks where sparse rewards are insufficient and dense rewards require elaborate design. In this work, we propose Autonomous Reinforcement learning for Complex HumanInformed Environments (ARCHIE), an unsupervised pipeline leveraging GPT-4, a pre-trained LLM, to generate reward functions directly from natural language task descriptions. The rewards are used to train RL agents in simulated environments, where we formalize the reward generation process to enhance feasibility. Additionally, GPT-4 automates the coding of task success criteria, creating a fully automated, one-shot procedure for translating human-readable text into deployable robot skills. Our approach is validated through extensive simulated experiments on single-arm and bi-manual manipulation tasks using an ABB YuMi collaborative robot, highlighting its practicality and effectiveness. Tasks are demonstrated on the real robot setup.


Achieving Operational Universality through a Turing Complete Chemputer

Gahler, Daniel, Thomas, Dean, Lach, Slawomir, Cronin, Leroy

arXiv.org Artificial Intelligence

The most fundamental abstraction underlying all modern computers is the Turing Machine, that is if any modern computer can simulate a Turing Machine, an equivalence which is called Turing completeness, it is theoretically possible to achieve any task that can be algorithmically described by executing a series of discrete unit operations. In chemistry, the ability to program chemical processes is demanding because it is hard to ensure that the process can be understood at a high level of abstraction, and then reduced to practice. Herein we exploit the concept of Turing completeness applied to robotic platforms for chemistry that can be used to synthesise complex molecules through unit operations that execute chemical processes using a chemically-aware programming language, XDL. We leverage the concept of computability by computers to synthesizability of chemical compounds by automated synthesis machines. The results of an interactive demonstration of Turing completeness using the colour gamut and conditional logic are presented and examples of chemical use-cases are discussed. Over 16.7 million combinations of Red, Green, Blue (RGB) colour space were binned into 5 discrete values and measured over 10 regions of interest (ROIs), affording 78 million possible states per step and served as a proxy for conceptual, chemical space exploration. This formal description establishes a formal framework in future chemical programming languages to ensure complex logic operations are expressed and executed correctly, with the possibility of error correction, in the automated and autonomous pursuit of increasingly complex molecules.


Re-expression of manual expertise through semi-automatic control of a teleoperated system

Landais, Erwann, Rezzoug, Nasser, Padois, Vincent

arXiv.org Artificial Intelligence

While the search for new solvents in the chemical industry is of uttermost importance with respect to environmental considerations, this domain remains strongly tied to highly manual and visual inspection tasks by human experts. As the manipulated chemicals may imply a critical danger (CMR substances), mechanical protection barrier are used (fume hoods, gloveboxes). This, in turn, can induce postural discomfort in the long term. Carrying out this task using a remotely controlled robot to reproduce the desired vial motions would alleviate these postural constraints. Nevertheless, the adoption of such a system will depend on its ability to transcribe the users' expertise. Particular attention must be paid to the intuitiveness of the system : transparency of the actions performed, relevance of the perceptual feedback, etc. and, in particular, the fidelity of the movements performed in relation to the user's commands. However, the extent of the rotational movements to be generated and the task interactivity complicates the problem both from the point of view of the motor capacities of industrial robots and for the transparency/responsiveness of the control.To tackle the problen of guaranteeing a secure and reactive expression of the manual characteristics of this task, we propose to separate the control of movement into two parts: control of the path (set of spatial poses) and of the trajectories associated with this path (speed, direction of travel along the path). The user can then partially control the robot's movements, by choosing the type of generic, secure path and modulating the trajectory performed on this path in real time. Although this drastically limits the possibilities for interaction, we assume that this teleoperated system can enable this type of observation task to be carried out as effectively as for direct manipulation. This hypothesis was tested through an experiment in which a reading task, less dangerous but with similar characteristics to the application task, had to be performed using different variants of trajectory modulation. This experiment consisted in reading words printed on four white capsules (dimensions 6 x 12 mm) placed into cylindrical vials ( dimensions 16 mm x 70 mm). Four randomly selected vials were tested by each variant. Firstly, users had to perform the task via direct handling, then under conditions secured by a protection barrier. Users were then invited to perform the task using different trajectory modulation variants (modulation and passive viewing of a pre-recorded video, modulation of the trajectory of a Franka-Emika Panda robot performing the task in real time in front of a monocular Logitech Brio 4K camera). After each trial of a variant, users evaluate different aspects of this variant (manual and visual performance, ease of use, acceptability of the interface) through a questionnaire. During the trials, various objective criteria are also measured (number and nature of interaction with the interface, time and degree of success in the task). This experiment was carried out with 37 subjects (age : 27$\pm$5, 20 females). The data recorded showed that the proportion of successes, as well as the subjects' perceptions of visual performance, comfort of use and acceptability of the interface, were similar and high for all the variants. This suggests that this task is indeed achievable via the proposed interface. However, data also showed that average task completion times when using the trajectory modulation variants were significantly higher than handling by hand variants, which implies that the proposed remote semi-automatic control procedure fails to achieve satisfactory performance regarding execution time. An interface allowing more reactive manipulation of the vial's movements seems necessary, and will be tested in a future experiment.