Agents
Meta-RAG on Large Codebases Using Code Summarization
Tawosi, Vali, Alamir, Salwa, Liu, Xiaomo, Veloso, Manuela
Large Language Model (LLM) systems have been at the forefront of applied Artificial Intelligence (AI) research in a multitude of domains. One such domain is software development, where researchers have pushed the automation of a number of code tasks through LLM agents. Software development is a complex ecosystem, that stretches far beyond code implementation and well into the realm of code maintenance. In this paper, we propose a multi-agent system to localize bugs in large pre-existing codebases using information retrieval and LLMs. Our system introduces a novel Retrieval Augmented Generation (RAG) approach, Meta-RAG, where we utilize summaries to condense codebases by an average of 79.8\%, into a compact, structured, natural language representation. We then use an LLM agent to determine which parts of the codebase are critical for bug resolution, i.e. bug localization. We demonstrate the usefulness of Meta-RAG through evaluation with the SWE-bench Lite dataset. Meta-RAG scores 84.67 % and 53.0 % for file-level and function-level correct localization rates, respectively, achieving state-of-the-art performance.
Streaming Generated Gaussian Process Experts for Online Learning and Control
Yang, Zewen, Zhang, Dongfa, Dai, Xiaobing, Yu, Fengyi, Zhang, Chi, Huang, Bingkun, Sadeghian, Hamid, Haddadin, Sami
Gaussian Processes (GPs), as a nonparametric learning method, offer flexible modeling capabilities and calibrated uncertainty quantification for function approximations. Additionally, GPs support online learning by efficiently incorporating new data with polynomial-time computation, making them well-suited for safety-critical dynamical systems that require rapid adaptation. However, the inference and online updates of exact GPs, when processing streaming data, incur cubic computation time and quadratic storage memory complexity, limiting their scalability to large datasets in real-time settings. In this paper, we propose a streaming kernel-induced progressively generated expert framework of Gaussian processes (SkyGP) that addresses both computational and memory constraints by maintaining a bounded set of experts, while inheriting the learning performance guarantees from exact Gaussian processes. Furthermore, two SkyGP variants are introduced, each tailored to a specific objective, either maximizing prediction accuracy (SkyGP-Dense) or improving computational efficiency (SkyGP-Fast). The effectiveness of SkyGP is validated through extensive benchmarks and real-time control experiments demonstrating its superior performance compared to state-of-the-art approaches.
HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agents
Liu, Yibin, Liang, Zhixuan, Chen, Zanxin, Chen, Tianxing, Hu, Mengkang, Dong, Wanxi, Xu, Congsheng, Han, Zhaoming, Qin, Yusen, Mu, Yao
Recent advances in multimodal large language models (MLLMs) have enabled richer perceptual grounding for code policy generation in embodied agents. However, most existing systems lack effective mechanisms to adaptively monitor policy execution and repair codes during task completion. In this work, we introduce HyCodePolicy, a hybrid language-based control framework that systematically integrates code synthesis, geometric grounding, perceptual monitoring, and iterative repair into a closed-loop programming cycle for embodied agents. Technically, given a natural language instruction, our system first decomposes it into subgoals and generates an initial executable program grounded in object-centric geometric primitives. The program is then executed in simulation, while a vision-language model (VLM) observes selected checkpoints to detect and localize execution failures and infer failure reasons. By fusing structured execution traces capturing program-level events with VLM-based perceptual feedback, HyCodePolicy infers failure causes and repairs programs. This hybrid dual feedback mechanism enables self-correcting program synthesis with minimal human supervision. Our results demonstrate that HyCodePolicy significantly improves the robustness and sample efficiency of robot manipulation policies, offering a scalable strategy for integrating multimodal reasoning into autonomous decision-making pipelines.
Finance as Extended Biology: Reciprocity as the Cognitive Substrate of Financial Behavior
A central challenge in economics and artificial intelligence is explaining how financial behaviors-such as credit, insurance, and trade-emerge without formal institutions. We argue that these functions are not products of institutional design, but structured extensions of a single behavioral substrate: reciprocity. Far from being a derived strategy, reciprocity served as the foundational logic of early human societies-governing the circulation of goods, regulation of obligation, and maintenance of long-term cooperation well before markets, money, or formal rules. Trade, commonly regarded as the origin of financial systems, is reframed here as the canonical form of reciprocity: simultaneous, symmetric, and partner-contingent. Building on this logic, we reconstruct four core financial functions-credit, insurance, token exchange, and investment-as expressions of the same underlying principle under varying conditions. By grounding financial behavior in minimal, simulateable dynamics of reciprocal interaction, this framework shifts the focus from institutional engineering to behavioral computation-offering a new foundation for modeling decentralized financial behavior in both human and artificial agents.
Reciprocity as the Foundational Substrate of Society: How Reciprocal Dynamics Scale into Social Systems
Prevailing accounts in both multi-agent AI and the social sciences explain social structure through top-down abstractions-such as institutions, norms, or trust-yet lack simulateable models of how such structures emerge from individual behavior. Ethnographic and archaeological evidence suggests that reciprocity served as the foundational mechanism of early human societies, enabling economic circulation, social cohesion, and interpersonal obligation long before the rise of formal institutions. Modern financial systems such as credit and currency can likewise be viewed as scalable extensions of reciprocity, formalizing exchange across time and anonymity. Building on this insight, we argue that reciprocity is not merely a local or primitive exchange heuristic, but the scalable substrate from which large-scale social structures can emerge. We propose a three-stage framework to model this emergence: reciprocal dynamics at the individual level, norm stabilization through shared expectations, and the construction of durable institutional patterns. This approach offers a cognitively minimal, behaviorally grounded foundation for simulating how large-scale social systems can emerge from decentralized reciprocal interaction.
The Cognitive Foundations of Economic Exchange: A Modular Framework Grounded in Behavioral Evidence
The origins of economic behavior remain unresolved-not only in the social sciences but also in AI, where dominant theories often rely on predefined incentives or institutional assumptions. Contrary to the longstanding myth of barter as the foundation of exchange, converging evidence from early human societies suggests that reciprocity-not barter-was the foundational economic logic, enabling communities to sustain exchange and social cohesion long before formal markets emerged. Yet despite its centrality, reciprocity lacks a simulateable and cognitively grounded account. Here, we introduce a minimal behavioral framework based on three empirically supported cognitive primitives-individual recognition, reciprocal credence, and cost--return sensitivity-that enable agents to participate in and sustain reciprocal exchange, laying the foundation for scalable economic behavior. These mechanisms scaffold the emergence of cooperation, proto-economic exchange, and institutional structure from the bottom up. By bridging insights from primatology, developmental psychology, and economic anthropology, this framework offers a unified substrate for modeling trust, coordination, and economic behavior in both human and artificial systems. For an interactive visualization of the framework, see: https://egil158.github.io/cogfoundations-econ/
Generic-to-Specific Reasoning and Learning for Scalable Ad Hoc Teamwork
Dodampegama, Hasra, Sridharan, Mohan
Consider one or more AI agents performing daily living tasks in collaboration with a human they have not worked with before. Figure 1 shows snapshots of a motivating scenario in which two AI agents (male, blue shirt; female, red dress) and a human agent (female, green top) are preparing breakfast and setting up a workstation. The agents (AI, human) have a limited view of the environment and do not communicate with each other, although each of them is aware of the state of the domain, including the location of teammates and the outcomes of their actions (e.g., change in location of an object moved by a teammate). The AI agents have to reason with different descriptions of domain knowledge and uncertainty that include qualitative statements ("eggs are usually in the fridge") and quantitative measures of uncertainty ("I am 90% sure I saw the eggs on the kitchen table"), adapting their actions to changes in the domain and teammates' behavior. These characteristics correspond to Ad Hoc T eamwork (AHT), which requires cooperation "on the fly" without prior coordination [1]; many practical problems such as disaster rescue are AHT problems. The state of the art in AHT has moved from using preset protocols that define specific actions to be performed in specific states, to methods that use a long history of prior experiences to build a deep network model of the behavior of other agents (or agent types) and optimize the ad hoc agent's behavior [2]. However, it is difficult to gather large datasets of different situations in complex domains. Also, these methods are opaque and make it difficult to revise the existing models over time. In a departure from existing work, we design an architecture for AHT that bridges knowledge-based and data-driven reasoning and learning, enabling an ad hoc agent to: Leverage the ability of a Large Language Model (LLM) to anticipate future high-level tasks to be completed, revising and adapting the LLM's output to domain-specific knowledge and experience; Perform non-monotonic logical reasoning with prior commonsense domain knowledge at different abstractions, and learned models predicting the behavior of other agents, toward achieving current and anticipated tasks as joint goals; and Rapidly identify the need for, learn, and revise the models predicting the behavior of each teammate to facilitate scalable collaboration in complex domains.
Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinement
Hao, Chao, Wang, Shuai, Zhou, Kaiwen
Graphical user interface (GUI) agents have shown promise in automating mobile tasks but still struggle with input redundancy and decision ambiguity. In this paper, we present \textbf{RecAgent}, an uncertainty-aware agent that addresses these issues through adaptive perception. We distinguish two types of uncertainty in GUI navigation: (1) perceptual uncertainty, caused by input redundancy and noise from comprehensive screen information, and (2) decision uncertainty, arising from ambiguous tasks and complex reasoning. To reduce perceptual uncertainty, RecAgent employs a component recommendation mechanism that identifies and focuses on the most relevant UI elements. For decision uncertainty, it uses an interactive module to request user feedback in ambiguous situations, enabling intent-aware decisions. These components are integrated into a unified framework that proactively reduces input complexity and reacts to high-uncertainty cases via human-in-the-loop refinement. Additionally, we propose a dataset called \textbf{ComplexAction} to evaluate the success rate of GUI agents in executing specified single-step actions within complex scenarios. Extensive experiments validate the effectiveness of our approach. The dataset and code will be available at https://github.com/Fanye12/RecAgent.
Transferring Expert Cognitive Models to Social Robots via Agentic Concept Bottleneck Models
Zhao, Xinyu, Tan, Zhen, Enisman, Maya, Seo, Minjae, Durantini, Marta R., Albarracin, Dolores, Chen, Tianlong
Successful group meetings, such as those implemented in group behavioral-change programs, work meetings, and other social contexts, must promote individual goal setting and execution while strengthening the social relationships within the group. Consequently, an ideal facilitator must be sensitive to the subtle dynamics of disengagement, difficulties with individual goal setting and execution, and interpersonal difficulties that signal a need for intervention. The challenges and cognitive load experienced by facilitators create a critical gap for an embodied technology that can interpret social exchanges while remaining aware of the needs of the individuals in the group and providing transparent recommendations that go beyond powerful but "black box" foundation models (FMs) that identify social cues. We address this important demand with a social robot co-facilitator that analyzes multimodal meeting data and provides discreet cues to the facilitator. The robot's reasoning is powered by an agentic concept bottleneck model (CBM), which makes decisions based on human-interpretable concepts like participant engagement and sentiments, ensuring transparency and trustworthiness. Our core contribution is a transfer learning framework that distills the broad social understanding of an FM into our specialized and transparent CBM. This concept-driven system significantly outperforms direct zero-shot FMs in predicting the need for intervention and enables real-time human correction of its reasoning. Critically, we demonstrate robust knowledge transfer: the model generalizes across different groups and successfully transfers the expertise of senior human facilitators to improve the performance of novices. By transferring an expert's cognitive model into an interpretable robotic partner, our work provides a powerful blueprint for augmenting human capabilities in complex social domains.
Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agents
Bao, Chongyu, Dai, Ruimin, Shen, Yangbo, Jian, Runyang, Zhang, Jinghan, Liu, Xiaolan, Liu, Kunpeng
Intelligent personal assistants (IPAs) such as Siri and Google Assistant are designed to enhance human capabilities and perform tasks on behalf of users. The emergence of LLM agents brings new opportunities for the development of IPAs. While responsive capabilities have been widely studied, proactive behaviors remain underexplored. Designing an IPA that is proactive, privacy-preserving, and capable of self-evolution remains a significant challenge. Designing such IPAs relies on the cognitive architecture of LLM agents. This work proposes Cognition Forest, a semantic structure designed to align cognitive modeling with system-level design. We unify cognitive architecture and system design into a self-reinforcing loop instead of treating them separately. Based on this principle, we present Galaxy, a framework that supports multidimensional interactions and personalized capability generation. Two cooperative agents are implemented based on Galaxy: KoRa, a cognition-enhanced generative agent that supports both responsive and proactive skills; and Kernel, a meta-cognition-based meta-agent that enables Galaxy's self-evolution and privacy preservation. Experimental results show that Galaxy outperforms multiple state-of-the-art benchmarks. Ablation studies and real-world interaction cases validate the effectiveness of Galaxy.