Agents
Jointly Assigning Processes to Machines and Generating Plans for Autonomous Mobile Robots in a Smart Factory
Leet, Christopher, Sciortino, Aidan, Koenig, Sven
-- A modern smart factory runs a manufacturing procedure using a collection of programmable machines. Typically, materials are ferried between these machines using a team of mobile robots. T o embed a manufacturing procedure in a smart factory, a factory operator must a) assign its processes to the smart factory's machines and b) determine how agents should carry materials between machines. Existing smart factory management systems solve the aforementioned problems sequentially, limiting the throughput that they can achieve. In this paper we introduce ACES, the Anytime Cyclic Embedding Solver, the first solver which jointly optimizes the assignment of processes to machines and the assignment of paths to agents. We evaluate ACES and show that it can scale to real industrial scenarios. I. INTRODUCTION Modern smart factories are designed to enable flexible manufacturing [1]. A flexible manufacturing system is a system which can produce a variety of different products with minimal reconfiguration [2]. Flexibility can improve a manufacturer's ability to customize products, reduce the time that it takes to fulfill new orders, and lower the costs of producing a new product. To permit flexible manufacturing, a smart factory needs the following two components: 1) Flexible Machines. Flexible machines are general-purpose machines such as CNC machines which can be programmed to carry out a range of manufacturing processes [4].
The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents
Duan, Yifan, Tang, Yihong, Bai, Xuefeng, Chen, Kehai, Li, Juntao, Zhang, Min
Large language models (LLMs) excel in both closed tasks (including problem-solving, and code generation) and open tasks (including creative writing), yet existing explanations for their capabilities lack connections to real-world human intelligence. To fill this gap, this paper systematically investigates LLM intelligence through the lens of ``human simulation'', addressing three core questions: (1) How do personality traits affect problem-solving in closed tasks? (2) How do traits shape creativity in open tasks? (3) How does single-agent performance influence multi-agent collaboration? By assigning Big Five personality traits to LLM agents and evaluating their performance in single- and multi-agent settings, we reveal that specific traits significantly influence reasoning accuracy (closed tasks) and creative output (open tasks). Furthermore, multi-agent systems exhibit collective intelligence distinct from individual capabilities, driven by distinguishing combinations of personalities. We demonstrate that LLMs inherently simulate human behavior through next-token prediction, mirroring human language, decision-making, and collaborative dynamics.
Acquiring Grounded Representations of Words with Situated Interactive Instruction
Mohan, Shiwali, Mininger, Aaron H., Kirk, James R., Laird, John E.
We present an approach for acquiring grounded representations of words from mixed-initiative, situated interactions with a human instructor. The work focuses on the acquisition of diverse types of knowledge including perceptual, semantic, and procedural knowledge along with learning grounded meanings. Interactive learning allows the agent to control its learning by requesting instructions about unknown concepts, making learning efficient. Our approach has been instantiated in Soar and has been evaluated on a table-top robotic arm capable of manipulating small objects.
Toward Fully Autonomous Flexible Chunk-Based Aerial Additive Manufacturing: Insights from Experimental Validation
Stamatopoulos, Marios-Nektarios, Haluska, Jakub, Small, Elias, Marroush, Jude, Banerjee, Avijit, Nikolakopoulos, George
A novel autonomous chunk-based aerial additive manufacturing framework is presented, supported with experimental demonstration advancing aerial 3D printing. An optimization-based decomposition algorithm transforms structures into sub-components, or chunks, treated as individual tasks coordinated via a dependency graph, ensuring sequential assignment to UA Vs considering inter-dependencies and printability constraints for seamless execution. A specially designed hexacopter equipped with a pressurized canister for lightweight expandable foam extrusion is utilized to deposit the material in a controlled manner. To further enhance precise execution of the printing, an offset-free Model Predictive Control mechanism is considered compensating reactively for disturbances and ground effect during execution. Additionally, an interlocking mechanism is introduced in the chunking process to enhance structural cohesion and improve layer adhesion. Extensive experiments demonstrate the framework's effectiveness in constructing precise structures of various shapes, while seamlessly adapting to practical challenges, proving its potential for a transformative leap in aerial robotic capability for autonomous construction. A video with the overall demonstration can be found here: https://youtu.be/WC1rLMLKEg4. Preprint submitted to Journal of Automation In Construction February 27, 2025 1. Introduction In recent times, ground breaking advancement in additive manufacturing, seamlessly integrated with autonomous robotics, are unlocking an exciting frontier in next generation construction and manufacturing process. Additive manufacturing has demonstrated a paradigm shift impact, addressing complex manufacturing processes with unprecedented precision and efficiency. Its transformative potential is becoming increasingly evident as it evolves and finds applications across a wide range of industries [1, 2, 3], while simultaneously paving the way for further innovations in the future. An intriguing development is its recent integration into the construction industry, capitalizing on its ability to automate construction processes, provide extensive design flexibility, and construct intricate structures designed using Computer-Aided Design (CAD) software [4, 5]. Numerous studies have demonstrated the design and deployment of large-scale robotic arms and gantry systems for printing building components and even entire houses using a variety of base materials [6]. A key advantage of such methods is their ability to adapt with high level of automation throughout the construction process, making them particularly well-suited for deployment in remote, inaccessible, and harsh environments[7, 8]. Notable examples include disaster-stricken areas, such as regions impacted by fires and earthquakes, where the rapid construction of shelters and basic infrastructure is imperative.
ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis
Li, Lei, Jia, Sen, Wang, Jianhao, An, Zhaochong, Li, Jiaang, Hwang, Jenq-Neng, Belongie, Serge
Advancements in Multimodal Large Language Models (MLLMs) have improved human motion understanding. However, these models remain constrained by their "instruct-only" nature, lacking interactivity and adaptability for diverse analytical perspectives. To address these challenges, we introduce ChatMotion, a multimodal multi-agent framework for human motion analysis. ChatMotion dynamically interprets user intent, decomposes complex tasks into meta-tasks, and activates specialized function modules for motion comprehension. It integrates multiple specialized modules, such as the MotionCore, to analyze human motion from various perspectives. Extensive experiments demonstrate ChatMotion's precision, adaptability, and user engagement for human motion understanding.
Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety
Shimanuki, Gabriel Kenji Godoy, Nascimento, Alexandre Moreira, Vismari, Lucio Flavio, Junior, Joao Batista Camargo, Junior, Jorge Rady de Almeida, Cugnasca, Paulo Sergio
In recent years, there has been significant development of autonomous vehicle (AV) technologies. However, despite the notable achievements of some industry players, a strong and appealing body of evidence that demonstrate AVs are actually safe is lacky, which could foster public distrust in this technology and further compromise the entire development of this industry, as well as related social impacts. To improve the safety of AVs, several techniques are proposed that use synthetic data in virtual simulation. In particular, the highest risk data, known as corner cases (CCs), are the most valuable for developing and testing AV controls, as they can expose and improve the weaknesses of these autonomous systems. In this context, the present paper presents a systematic literature review aiming to comprehensively analyze methodologies for CC identifi cation and generation, also pointing out current gaps and further implications of synthetic data for AV safety and reliability. Based on a selection criteria, 110 studies were picked from an initial sample of 1673 papers. These selected paper were mapped into multiple categories to answer eight inter-linked research questions. It concludes with the recommendation of a more integrated approach focused on safe development among all stakeholders, with active collaboration between industry, academia and regulatory bodies.
CAMETA: Conflict-Aware Multi-Agent Estimated Time of Arrival Prediction for Mobile Robots
Sejersen, Jonas le Fevre, Kayacan, Erdal
-- This study presents the conflict-aware multi-agent estimated time of arrival (CAMET A) framework, a novel approach for predicting the arrival times of multiple agents in unstructured environments without predefined road infrastructure. The CAMET A framework consists of three components: a path planning layer generating potential path suggestions, a multi-agent ET A prediction layer predicting the arrival times for all agents based on the paths, and lastly, a path selection layer that calculates the accumulated cost and selects the best path. The novelty of the CAMET A framework lies in the heterogeneous map representation and the heterogeneous graph neural network architecture. As a result of the proposed novel structure, CAMET A improves the generalization capability compared to the state-of-the-art methods that rely on structured road infrastructure and historical data. The simulation results demonstrate the efficiency and efficacy of the multi-agent ET A prediction layer, with a mean average percentage error improvement of 29.5% and 44% when compared to a traditional path planning method ( A The performance of the CAMET A framework shows significant improvements in terms of robustness to noise and conflicts as well as determining proficient routes compared to state-of-the-art multi-agent path planners. Multi-agent path finding (MAPF) is the problem of generating valid paths for multiple agents while avoiding conflicts. This problem is highly relevant in many real-world applications, such as logistics, transportation, and robotics, where multiple agents must operate in a shared environment. MAPF is a challenging problem due to the need to find paths that avoid conflicts while minimizing the overall travel time for all agents. Many state-of-the-art MAPF solvers [1, 2, 3] employ various techniques to find a set of conflict-free paths on graphs representing the environment and the agents. However, a common limitation of these solvers is that they tend to generate tightly planned and coordinated paths. Therefore, the agents are expected to follow the exact path prescribed by the solver, which can lead to problems when applied to real-world systems with imperfect plan execution and uncertainties in the environment. This work introduces a conflict-aware multi-agent estimated time of arrival (CAMET A) for indoor autonomous mobile robot (AMR) applications that operate in time-constrained scenarios.
Delayed-Decision Motion Planning in the Presence of Multiple Predictions
Isele, David, Anon, Alexandre Miranda, Tariq, Faizan M., Yeh, Goro, Singh, Avinash, Bae, Sangjae
-- Reliable automated driving technology is challenged by various sources of uncertainties, in particular, behavioral uncertainties of traffic agents. It is common for traffic agents to have intentions that are unknown to others, leaving an automated driving car to reason over multiple possible behaviors. This paper formalizes a behavior planning scheme in the presence of multiple possible futures with corresponding probabilities. We present a maximum entropy formulation and show how, under certain assumptions, this allows delayed decision-making to improve safety. The general formulation is then turned into a model predictive control formulation, which is solved as a quadratic program or a set of quadratic programs. We discuss implementation details for improving computation and verify operation in simulation and on a mobile robot. Prediction technology continues to advance, and multiple prediction outputs are now a staple of state-of-the-art prediction methods [1]-[6]. This paper examines how an autonomous driving (AD) agent can utilize multiple predictions in the behavior planning process. In the context of this work, behavior planning corresponds to the combined task of decision-making and trajectory planning. Consider the scenario depicted in Figure 1. A pedestrian walks along a road and will likely continue straight (with 80% probability), but the pedestrian is positioned close to the street, indicating that they might turn to cross the street (with 20% probability). Selecting the most probable sequence of events results in an overly aggressive and risky behavior - we assume they will not cross and are wrong 20% of the time.
Close-Proximity Satellite Operations through Deep Reinforcement Learning and Terrestrial Testing Environments
Lei, Henry, Aurand, Joshua, Lippay, Zachary S., Phillips, Sean
With the increasingly congested and contested space environment, safe and effective satellite operation has become increasingly challenging. As a result, there is growing interest in autonomous satellite capabilities, with common machine learning techniques gaining attention for their potential to address complex decision-making in the space domain. However, the "black-box" nature of many of these methods results in difficulty understanding the model's input/output relationship and more specifically its sensitivity to environmental disturbances, sensor noise, and control intervention. This paper explores the use of Deep Reinforcement Learning (DRL) for satellite control in multi-agent inspection tasks. The Local Intelligent Network of Collaborative Satellites (LINCS) Lab is used to test the performance of these control algorithms across different environments, from simulations to real-world quadrotor UAV hardware, with a particular focus on understanding their behavior and potential degradation in performance when deployed beyond the training environment.
On Benchmarking Human-Like Intelligence in Machines
Ying, Lance, Collins, Katherine M., Wong, Lionel, Sucholutsky, Ilia, Liu, Ryan, Weller, Adrian, Shu, Tianmin, Griffiths, Thomas L., Tenenbaum, Joshua B.
Recent benchmark studies have claimed that AI has approached or even surpassed human-level performances on various cognitive tasks. However, this position paper argues that current AI evaluation paradigms are insufficient for assessing human-like cognitive capabilities. We identify a set of key shortcomings: a lack of human-validated labels, inadequate representation of human response variability and uncertainty, and reliance on simplified and ecologically-invalid tasks. We support our claims by conducting a human evaluation study on ten existing AI benchmarks, suggesting significant biases and flaws in task and label designs. To address these limitations, we propose five concrete recommendations for developing future benchmarks that will enable more rigorous and meaningful evaluations of human-like cognitive capacities in AI with various implications for such AI applications.