Woburn
Efficient Learning by Directed Acyclic Graph For Resource Constrained Prediction
Joseph Wang, Kirill Trapeznikov, Venkatesh Saligrama
We study the problem of reducing test-time acquisition costs in classification systems. Our goal is to learn decision rules that adaptively select sensors for each example as necessary to make a confident prediction. We model our system as a directed acyclic graph (DAG) where internal nodes correspond to sensor subsets and decision functions at each node choose whether to acquire a new sensor or classify using the available measurements. This problem can be posed as an empirical risk minimization over training data. Rather than jointly optimizing such a highly coupled and non-convex problem over all decision nodes, we propose an efficient algorithm motivated by dynamic programming. We learn node policies in the DAG by reducing the global objective to a series of cost sensitive learning problems. Our approach is computationally efficient and has proven guarantees of convergence to the optimal system for a fixed architecture. In addition, we present an extension to map other budgeted learning problems with large number of sensors to our DAG architecture and demonstrate empirical performance exceeding state-of-the-art algorithms for data composed of both few and many sensors.
Birds of a Different Feather Flock Together: Exploring Opportunities and Challenges in Animal-Human-Machine Teaming
Cohen, Myke C., Grimm, David A., Mirsky, Reuth, Yin, Xiaoyun
Birds of a Different Feather Flock Together: Exploring Opportunities and Challenges in Animal-Human-Machine Teaming Myke C. Cohen 1,2, David A. Grimm 3, Reuth Mirsky 4, and Xiaoyun Yin 1 1 Arizona State University, Mesa, AZ 2 Aptima, Inc., Woburn, MA 3 Georgia Institute of Technology, Atlanta, GA 4 Tufts University, Medford, MA Abstract Animal-Human-Machine (AHM) teams are a type of hybrid intelligence system wherein interactions between a human, AI-enabled machine, and animal members can result in unique capabilities greater than the sum of their parts. This paper calls for a systematic approach to studying the design of AHM team structures to optimize performance and overcome limitations in various applied settings. We consider the challenges and opportunities in investigating the synergistic potential of AHM team members by introducing a set of dimensions of AHM team functioning to effectively utilize each member's strengths while compensating for individual weaknesses. Using three representative examples of such teams--security screening, search-and-rescue, and guide dogs--the paper illustrates how AHM teams can tackle complex tasks. We conclude with open research directions that this multidimensional approach presents for studying hybrid human-AI systems beyond AHM teams. Keywords: multi-agent systems, animal-human-machine teaming, functional allocation 1 Introduction Consider a Blind or Visually Impaired (BVI) person training to be assisted by a guide dog. When the pair reaches an obstacle along their path, they 1 arXiv:2504.13973v1
Passive Heart Rate Monitoring During Smartphone Use in Everyday Life
Liao, Shun, Di Achille, Paolo, Wu, Jiang, Borac, Silviu, Wang, Jonathan, Liu, Xin, Teasley, Eric, Cai, Lawrence, Liu, Yun, McDuff, Daniel, Su, Hao-Wei, Winslow, Brent, Pathak, Anupam, Patel, Shwetak, Taylor, James A., Rogers, Jameson K., Poh, Ming-Zher
Resting heart rate (RHR) is an important biomarker of cardiovascular health and mortality, but tracking it longitudinally generally requires a wearable device, limiting its availability. We present PHRM, a deep learning system for passive heart rate (HR) and RHR measurements during everyday smartphone use, using facial video-based photoplethysmography. Our system was developed using 225,773 videos from 495 participants and validated on 185,970 videos from 205 participants in laboratory and free-living conditions, representing the largest validation study of its kind. Compared to reference electrocardiogram, PHRM achieved a mean absolute percentage error (MAPE) < 10% for HR measurements across three skin tone groups of light, medium and dark pigmentation; MAPE for each skin tone group was non-inferior versus the others. Daily RHR measured by PHRM had a mean absolute error < 5 bpm compared to a wearable HR tracker, and was associated with known risk factors. These results highlight the potential of smartphones to enable passive and equitable heart health monitoring.
Exploratory Models of Human-AI Teams: Leveraging Human Digital Twins to Investigate Trust Development
Nguyen, Daniel, Cohen, Myke C., Kao, Hsien-Te, Engberson, Grant, Penafiel, Louis, Lynch, Spencer, Volkova, Svitlana
As human-agent teaming (HAT) research continues to grow, computational methods for modeling HAT behaviors and measuring HAT effectiveness also continue to develop. One rising method involves the use of human digital twins (HDT) to approximate human behaviors and socio-emotional-cognitive reactions to AI-driven agent team members. In this paper, we address three research questions relating to the use of digital twins for modeling trust in HATs. First, to address the question of how we can appropriately model and operationalize HAT trust through HDT HAT experiments, we conducted causal analytics of team communication data to understand the impact of empathy, socio-cognitive, and emotional constructs on trust formation. Additionally, we reflect on the current state of the HAT trust science to discuss characteristics of HAT trust that must be replicable by a HDT such as individual differences in trust tendencies, emergent trust patterns, and appropriate measurement of these characteristics over time. Second, to address the question of how valid measures of HDT trust are for approximating human trust in HATs, we discuss the properties of HDT trust: self-report measures, interaction-based measures, and compliance type behavioral measures. Additionally, we share results of preliminary simulations comparing different LLM models for generating HDT communications and analyze their ability to replicate human-like trust dynamics. Third, to address how HAT experimental manipulations will extend to human digital twin studies, we share experimental design focusing on propensity to trust for HDTs vs. transparency and competency-based trust for AI agents.
Encoding Agent Trajectories as Representations with Sequence Transformers
Tsiligkaridis, Athanasios, Kalinowski, Nicholas, Li, Zhongheng, Hou, Elizabeth
Spatiotemporal data faces many analogous challenges to natural language text including the ordering of locations (words) in a sequence, long range dependencies between locations, and locations having multiple meanings. In this work, we propose a novel model for representing high dimensional spatiotemporal trajectories as sequences of discrete locations and encoding them with a Transformer-based neural network architecture. Similar to language models, our Sequence Transformer for Agent Representation Encodings (STARE) model can learn representations and structure in trajectory data through both supervisory tasks (e.g., classification), and self-supervisory tasks (e.g., masked modelling). We present experimental results on various synthetic and real trajectory datasets and show that our proposed model can learn meaningful encodings that are useful for many downstream tasks including discriminating between labels and indicating similarity between locations. Using these encodings, we also learn relationships between agents and locations present in spatiotemporal data.
Simulation-based Scenario Generation for Robust Hybrid AI for Autonomy
Keno, Hambisa, Pioch, Nicholas J., Guagliano, Christopher, Chung, Timothy H.
Application of Unmanned Aerial Vehicles (UAVs) in search and rescue, emergency management, and law enforcement has gained traction with the advent of low-cost platforms and sensor payloads. The emergence of hybrid neural and symbolic AI approaches for complex reasoning is expected to further push the boundaries of these applications with decreasing levels of human intervention. However, current UAV simulation environments lack semantic context suited to this hybrid approach. To address this gap, HAMERITT (Hybrid Ai Mission Environment for RapId Training and Testing) provides a simulation-based autonomy software framework that supports the training, testing and assurance of neuro-symbolic algorithms for autonomous maneuver and perception reasoning. HAMERITT includes scenario generation capabilities that offer mission-relevant contextual symbolic information in addition to raw sensor data. Scenarios include symbolic descriptions for entities of interest and their relations to scene elements, as well as spatial-temporal constraints in the form of time-bounded areas of interest with prior probabilities and restricted zones within those areas. HAMERITT also features support for training distinct algorithm threads for maneuver vs. perception within an end-to-end mission run. Future work includes improving scenario realism and scaling symbolic context generation through automated workflow.
Kinesthetic Teaching in Robotics: a Mixed Reality Approach
Macci`o, Simone, Shaaban, Mohamad, Carf`ฤฑ, Alessandro, Mastrogiovanni, Fulvio
Abstract-- As collaborative robots become more common in manufacturing scenarios and adopted in hybrid human-robot teams, we should develop new interaction and communication strategies to ensure smooth collaboration between agents. In this paper, we propose a novel communicative interface that uses Mixed Reality as a medium to perform Kinesthetic Teaching (KT) on any robotic platform. We evaluate our proposed approach in a user study involving multiple subjects and two different robots, comparing traditional physical KT with holographic-based KT through user experience questionnaires and task-related metrics. Index Terms-- Human-Robot Interaction, Mixed Reality, Kinesthetic Teaching, Software Architecture. In smart factories, robots are expected to coexist and work alongside humans rather than replace them.
Temporal assessment of malicious behaviors: application to turnout field data monitoring
Abdellaoui, Sara, Dumitrescu, Emil, Escudero, Cรฉdric, Zamaรฏ, Eric
This information was projected on the life cycle of the Their distributed communicating nature makes them vulnerable turnout according to time aging and operation aging to cyberattacks [2]. The security of CPS has criteria in order to compute a cyberthreat likelihood for emerged as a complex problem, after discovering the each current curve observed. Maintenance operators use Stuxnet malware [3] that targeted the Iranian industrial the estimated likelihood to assess the authenticity of each control system.
Efficient Learning by Directed Acyclic Graph For Resource Constrained Prediction
We study the problem of reducing test-time acquisition costs in classification systems. Our goal is to learn decision rules that adaptively select sensors for each example as necessary to make a confident prediction. We model our system as a directed acyclic graph (DAG) where internal nodes correspond to sensor subsets and decision functions at each node choose whether to acquire a new sensor or classify using the available measurements. This problem can be posed as an empirical risk minimization over training data. Rather than jointly optimizing such a highly coupled and non-convex problem over all decision nodes, we propose an efficient algorithm motivated by dynamic programming. We learn node policies in the DAG by reducing the global objective to a series of cost sensitive learning problems. Our approach is computationally efficient and has proven guarantees of convergence to the optimal system for a fixed architecture. In addition, we present an extension to map other budgeted learning problems with large number of sensors to our DAG architecture and demonstrate empirical performance exceeding state-of-the-art algorithms for data composed of both few and many sensors.
Scalable Semantic Non-Markovian Simulation Proxy for Reinforcement Learning
Mukherji, Kaustuv, Parkar, Devendra, Pokala, Lahari, Aditya, Dyuman, Shakarian, Paulo, Dorman, Clark
Recent advances in reinforcement learning (RL) have shown much promise across a variety of applications. However, issues such as scalability, explainability, and Markovian assumptions limit its applicability in certain domains. We observe that many of these shortcomings emanate from the simulator as opposed to the RL training algorithms themselves. As such, we propose a semantic proxy for simulation based on a temporal extension to annotated logic. In comparison with two high-fidelity simulators, we show up to three orders of magnitude speed-up while preserving the quality of policy learned. In addition, we show the ability to model and leverage non-Markovian dynamics and instantaneous actions while providing an explainable trace describing the outcomes of the agent actions.