Goto

Collaborating Authors

 intervention rate


Gated Uncertainty-Aware Runtime Dual Invariants for Neural Signal-Controlled Robotics

Kim, Tasha, Jones, Oiwi Parker

arXiv.org Artificial Intelligence

Safety-critical assistive systems that directly decode user intent from neural signals require rigorous guarantees of reliability and trust. We present GUARDIAN (Gated Uncertainty-Aware Runtime Dual Invariants), a framework for real-time neuro-symbolic verification for neural signal-controlled robotics. GUARDIAN enforces both logical safety and physiological trust by coupling confidence-calibrated brain signal decoding with symbolic goal grounding and dual-layer runtime monitoring. On the BNCI2014 motor imagery electroencephalogram (EEG) dataset with 9 subjects and 5,184 trials, the system performs at a high safety rate of 94-97% even with lightweight decoder architectures with low test accuracies (27-46%) and high ECE confidence miscalibration (0.22-0.41). We demonstrate 1.7x correct interventions in simulated noise testing versus at baseline. The monitor operates at 100Hz and sub-millisecond decision latency, making it practically viable for closed-loop neural signal-based systems. Across 21 ablation results, GUARDIAN exhibits a graduated response to signal degradation, and produces auditable traces from intent, plan to action, helping to link neural evidence to verifiable robot action.


$How^{2}$: How to learn from procedural How-to questions

Dagan, Gautier, Keller, Frank, Lascarides, Alex

arXiv.org Artificial Intelligence

An agent facing a planning problem can use answers to how-to questions to reduce uncertainty and fill knowledge gaps, helping it solve both current and future tasks. However, their open ended nature, where valid answers to "How do I X?" range from executable actions to high-level descriptions of X's sub-goals, makes them challenging for AI agents to ask, and for AI experts to answer, in ways that support efficient planning. We introduce $How^{2}$, a memory agent framework that enables agents to ask how-to questions, store the answers, and reuse them for lifelong learning in interactive environments. We evaluate our approach in Plancraft, a Minecraft crafting environment, where agents must complete an assembly task by manipulating inventory items. Using teacher models that answer at varying levels of abstraction, from executable action sequences to high-level subgoal descriptions, we show that lifelong learning agents benefit most from answers that are abstracted and decoupled from the current state. $How^{2}$ offers a way for LLM-based agents to improve their planning capabilities over time by asking questions in interactive environments.


Fair Classification by Direct Intervention on Operating Characteristics

Jiang, Kevin, Dobriban, Edgar

arXiv.org Machine Learning

We develop new classifiers under group fairness in the attribute-aware setting for binary classification with multiple group fairness constraints (e.g., demographic parity (DP), equalized odds (EO), and predictive parity (PP)). We propose a novel approach, applicable to linear fractional constraints, based on directly intervening on the operating characteristics of a pre-trained base classifier, by (i) identifying optimal operating characteristics using the base classifier's group-wise ROC convex hulls and (ii) post-processing the base classifier to match those targets. As practical post-processors, we consider randomizing a mixture of group-wise thresholding rules subject to minimizing the expected number of interventions. We further extend our approach to handle multiple protected attributes and multiple linear fractional constraints. On standard datasets (COMPAS and ACSIncome), our methods simultaneously satisfy approximate DP, EO, and PP with few interventions and a near-oracle drop in accuracy; comparing favorably to previous methods.


Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach

Wei, Peng, Ragbir, Prabhash, Vougioukas, Stavros G., Kong, Zhaodan

arXiv.org Artificial Intelligence

Autonomous unmanned aerial vehicle (UAV) navigation in orchards presents significant challenges due to obstacles and GPS-deprived environments. In this work, we introduce a learning-based approach to achieve vision-based navigation of UAVs within orchard rows. Our method employs a variational autoencoder (VAE)-based controller, trained with an intervention-based learning framework that allows the UAV to learn a visuomotor policy from human experience. Field experiments demonstrate that after only a few iterations of training, the proposed VAE-based controller can autonomously navigate the UAV based on a front-mounted camera stream. The controller exhibits strong obstacle avoidance performance, achieves longer flying distances with less human assistance, and outperforms existing algorithms. Furthermore, we show that the policy generalizes effectively to novel environments and maintains competitive performance across varying conditions and speeds. This research not only advances UAV autonomy but also holds significant potential for precision agriculture, improving efficiency in orchard monitoring and management. Introduction Unmanned aerial vehicle (UAV) technology has made significant progress in recent years, particularly for applications in agriculture. The ability to navigate within orchard rows allows UAVs to perform tasks such as crop inspection and yield estimation (Zhang et al., 2021). This capability provides a valuable tool for remote sensing and precision agriculture (Chen et al., 2022), leading to more efficient and improved orchard management. However, most existing UAVs still depend on GPS for navigation in agricultural settings. This reliance limits their ability to operate in confined orchard rows, where dense tree canopies can block GPS signals. Additionally, in environments with unknown obstacles, such as tree branches in orchard rows, human pilots are frequently queried to provide avoidance maneuvers, which significantly increases their workload. The ability to navigate autonomously and safely in orchard scenes with weak GPS signals and obstacles presents several challenges and largely hinders the deployment of UAVs in orchard operations. Corresponding author Email address: zdkong@ucdavis.edu The view of the onboard camera is provided. When the GPS signal is attenuated, the UAV may rely on exteroceptive sensors to sense the environment and navigate. Advanced techniques to enable UAV autonomous operations without GPS include: 1) lidar-based, and 2) camera-based approaches.


Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation

Pan, Cheng, Cheng, Hung Hon, Hughes, Josie

arXiv.org Artificial Intelligence

Teleoperated robotic manipulators enable the collection of demonstration data, which can be used to train control policies through imitation learning. However, such methods can require significant amounts of training data to develop robust policies or adapt them to new and unseen tasks. While expert feedback can significantly enhance policy performance, providing continuous feedback can be cognitively demanding and time-consuming for experts. To address this challenge, we propose to use a cable-driven teleoperation system which can provide spatial corrections with 6 degree of freedom to the trajectories generated by a policy model. Specifically, we propose a correction method termed Decaying Relative Correction (DRC) which is based upon the spatial offset vector provided by the expert and exists temporarily, and which reduces the intervention steps required by an expert. Our results demonstrate that DRC reduces the required expert intervention rate by 30\% compared to a standard absolute corrective method. Furthermore, we show that integrating DRC within an online imitation learning framework rapidly increases the success rate of manipulation tasks such as raspberry harvesting and cloth wiping.


On Optimizing Interventions in Shared Autonomy

Tan, Weihao, Koleczek, David, Pradhan, Siddhant, Perello, Nicholas, Chettiar, Vivek, Rohra, Vishal, Rajaram, Aaslesha, Srinivasan, Soundararajan, Hossain, H M Sajjad, Chandak, Yash

arXiv.org Artificial Intelligence

Shared autonomy refers to approaches for enabling an autonomous agent to collaborate with a human with the aim of improving human performance. However, besides improving performance, it may often also be beneficial that the agent concurrently accounts for preserving the user's experience or satisfaction of collaboration. In order to address this additional goal, we examine approaches for improving the user experience by constraining the number of interventions by the autonomous agent. We propose two model-free reinforcement learning methods that can account for both hard and soft constraints on the number of interventions. We show that not only does our method outperform the existing baseline, but also eliminates the need to manually tune a black-box hyperparameter for controlling the level of assistance. We also provide an in-depth analysis of intervention scenarios in order to further illuminate system understanding.


Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning

Chen, Min, Nikolaidis, Stefanos, Soh, Harold, Hsu, David, Srinivasa, Siddhartha

arXiv.org Artificial Intelligence

Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human trust, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). In our studies, the robot builds human trust by manipulating low-risk objects first. Interestingly, the robot sometimes fails intentionally in order to modulate human trust and achieve the best team performance. These results show that the trust-POMDP calibrates trust to improve human-robot team performance over the long term. Further, they highlight that maximizing trust alone does not always lead to the best performance.