Goto

Collaborating Authors

 autogen


Structuring Collective Action with LLM-Guided Evolution: From Ill-Structured Problems to Executable Heuristics

arXiv.org Artificial Intelligence

Collective action problems, which require aligning individual incentives with collective goals, are classic examples of Ill-Structured Problems (ISPs). For an individual agent, the causal links between local actions and global outcomes are unclear, stakeholder objectives often conflict, and no single, clear algorithm can bridge micro-level choices with macro-level welfare. We present ECHO-MIMIC, a general computational framework that converts this global complexity into a tractable, Well-Structured Problem (WSP) for each agent by discovering executable heuristics and persuasive rationales. The framework operates in two stages: ECHO (Evolutionary Crafting of Heuristics from Outcomes) evolves snippets of Python code that encode candidate behavioral policies, while MIMIC (Mechanism Inference \& Messaging for Individual-to-Collective Alignment) evolves companion natural language messages that motivate agents to adopt those policies. Both phases employ a large-language-model-driven evolutionary search: the LLM proposes diverse and context-aware code or text variants, while population-level selection retains those that maximize collective performance in a simulated environment. We demonstrate this framework on two distinct ISPs: a canonical agricultural landscape management problem and a carbon-aware EV charging time slot usage problem. Results show that ECHO-MIMIC discovers high-performing heuristics compared to baselines and crafts tailored messages that successfully align simulated agent behavior with system-level goals. By coupling algorithmic rule discovery with tailored communication, ECHO-MIMIC transforms the cognitive burden of collective action into a implementable set of agent-level instructions, making previously ill-structured problems solvable in practice and opening a new path toward scalable, adaptive policy design.


Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations

arXiv.org Artificial Intelligence

As Large Language Model (LLM) agents become increasingly capable of automating complex, multi-step tasks, the need for robust, secure, and predictable architectural patterns is paramount. This paper provides a comprehensive guide to the ``Plan-then-Execute'' (P-t-E) pattern, an agentic design that separates strategic planning from tactical execution. We explore the foundational principles of P-t-E, detailing its core components - the Planner and the Executor - and its architectural advantages in predictability, cost-efficiency, and reasoning quality over reactive patterns like ReAct (Reason + Act). A central focus is placed on the security implications of this design, particularly its inherent resilience to indirect prompt injection attacks by establishing control-flow integrity. We argue that while P-t-E provides a strong foundation, a defense-in-depth strategy is necessary, and we detail essential complementary controls such as the Principle of Least Privilege, task-scoped tool access, and sandboxed code execution. To make these principles actionable, this guide provides detailed implementation blueprints and working code references for three leading agentic frameworks: LangChain (via LangGraph), CrewAI, and AutoGen. Each framework's approach to implementing the P-t-E pattern is analyzed, highlighting unique features like LangGraph's stateful graphs for re-planning, CrewAI's declarative tool scoping for security, and AutoGen's built-in Docker sandboxing. Finally, we discuss advanced patterns, including dynamic re-planning loops, parallel execution with Directed Acyclic Graphs (DAGs), and the critical role of Human-in-the-Loop (HITL) verification, to offer a complete strategic blueprint for architects, developers, and security engineers aiming to build production-grade, resilient, and trustworthy LLM agents.


Situating AI Agents in their World: Aspective Agentic AI for Dynamic Partially Observable Information Systems

arXiv.org Artificial Intelligence

Agentic LLM AI agents are often little more than autonomous chatbots: actors following scripts, often controlled by an unreliable director. This work introduces a bottom-up framework that situates AI agents in their environment, with all behaviors triggered by changes in their environments. It introduces the notion of aspects, similar to the idea of umwelt, where sets of agents perceive their environment differently to each other, enabling clearer control of information. We provide an illustrative implementation and show that compared to a typical architecture, which leaks up to 83% of the time, aspective agentic AI enables zero information leakage. We anticipate that this concept of specialist agents working efficiently in their own information niches can provide improvements to both security and efficiency.


Agentic AI Frameworks: Architectures, Protocols, and Design Challenges

arXiv.org Artificial Intelligence

Aspect Traditional AI agents Modern agentic AI systems (LLM-based agents) Definition Autonomous entities with fixed sensing/acting loops; limited by static rules or models Autonomous reasoning systems using LLMs with dynamic behavior, tool orchestration, and context-awarenessAutonomy Limited autonomy; often dependent on human input or predefined instructions High autonomy; capable of independently performing complex and extended tasks Goal Management Focused on single, static goals or fixed task planning Capable of managing multiple, evolving, and nested goals adaptivelyArchitecture Rule-based or BDI (Belief-Desire-Intention) models; monolithic design Modular architecture centered on LLMs, with components for memory, tools, context injection, and rolesAdaptability Suited to controlled, predictable environments; poor generalization Designed for open, dynamic, and unpredictable environmentsDecision-Making Deterministic or rule-based logic; symbolic reasoning Context-sensitive, probabilistic reasoning with adaptive planning and self-reflection Learning Mechanism Rule-based or supervised learning with limited updates Self-supervised and reinforcement learning; continual fine-tuning possible Context Handling Static or manually coded states and rules Dynamic context injection via agent protocols (e.g., MCP, A2A) and runtime awareness Communication Message-passing via ACL or KQML Real-time, event-driven collaboration; natural language interfacesTool Use Limited or predefined tools and actions Dynamic tool invocation, chaining, and API calling based on contextMemory Optional, often hardcoded or task-specific Integrated memory systems supporting long-and short-term information retention


AutoGen Driven Multi Agent Framework for Iterative Crime Data Analysis and Prediction

arXiv.org Artificial Intelligence

Figure 4: P lot over 100 epochs with 3 - Agents F. Ablation Study - Impact of the LearningOptimizerAgent To quantify the OptimizerAgent's effect on the system, we conducted an ablation study that set up two different configurations. Baseline (3 - Agent Framework): CrimeAnalysisAssistant, FeedbackAgent, and CrimePredictorAgent. Extended (4 - Agent Framework): All of the above, with the OptimizerAgent that could oversee and control how the other agents worked. Both settings were tested using the same protocol, working with the same data for 100 epochs and evaluated according to the already mentioned metrics described in Section V - B. Importantly, during the extended framework tests the OptimizerAgent did not have access to the ground truth and its actions reflected those of a real - world supervisor trying to be efficient with resources . The main aim was to bring more stability and better learning curve using our framework LUCID - MA. Table 2: 4 - Aegnts Observed Improvement Metric Baseline (3 agents) With OptimizerAgent Improvement CrimeAnalysis Assistant Final Score 0.94 0.96 +0.02 FeedbackAgent Final Score 0.89 0.92 +0.03 CrimePredictorAgent Final Score 0.85 0.91 +0.06 Avg. Redundancy Across Epochs 14.2% 6.8% - 7.4% Using the OptimizerAgent resulted in a marked increase in the variety and quality of final system outputs . Visual Result: The final plot demonstrates that agent - level meta - control, As a result, the model exhibits higher consistency, greater variety in its results and more reliable improvement over time -- all accomplished without any need for further model fine - tuning. Figure 5: P lot over 100 epochs with 4 - Agents In addition to standard performance comparison metrics, our system portrayed advanced behavioral dynamics pointing to the pre sence of emergent intelligence capabilities which we delve into in the next section in great detail.


Who's to Blame When AI Agents Screw Up?

WIRED

Over the past year, veteran software engineer Jay Prakash Thakur has spent his nights and weekends prototyping AI agents that could, in the near future, order meals and engineer mobile apps almost entirely on their own. His agents, while surprisingly capable, have also exposed new legal questions that await companies trying to capitalize on Silicon Valley's hottest new technology. Agents are AI programs that can act mostly independently, allowing companies to automate tasks such as answering customer questions or paying invoices. While ChatGPT and similar chatbots can draft emails or analyze bills upon request, Microsoft and other tech giants expect that agents will tackle more complex functions--and most importantly, do it with little human oversight. The tech industry's most ambitious plans involve multi-agent systems, with dozens of agents someday teaming up to replace entire workforces.


Chatbot Teamwork Makes the AI Dream Work

WIRED

Turning to a friend or coworker can make tricky problems easier to tackle. Now it looks like having AI chatbots team up with each other can make them more effective. I've been playing this week with AutoGen, an open source software framework for AI agent collaboration developed by researchers at Microsoft and academics at Pennsylvania State University, the University of Washington, and Xidian University in China. The software taps OpenAI's large language model GPT-4 to let you create multiple AI agents with different personas, roles, and objectives that can be prompted to solve specific problems. To put the idea of AI collaboration to the test, I had two AI agents work together on a plan for how to write about AI collaboration.


Assessing and Verifying Task Utility in LLM-Powered Applications

arXiv.org Artificial Intelligence

The rapid development of Large Language Models (LLMs) has led to a surge in applications that facilitate collaboration among multiple agents, assisting humans in their daily tasks. However, a significant gap remains in assessing to what extent LLM-powered applications genuinely enhance user experience and task execution efficiency. This highlights the need to verify utility of LLM-powered applications, particularly by ensuring alignment between the application's functionality and end-user needs. We introduce AgentEval, a novel framework designed to simplify the utility verification process by automatically proposing a set of criteria tailored to the unique purpose of any given application. This allows for a comprehensive assessment, quantifying the utility of an application against the suggested criteria. We present a comprehensive analysis of the effectiveness and robustness of AgentEval for two open source datasets including Math Problem solving and ALFWorld House-hold related tasks. For reproducibility purposes, we make the data, code and all the logs publicly available at https://bit.ly/3w3yKcS .


Towards better Human-Agent Alignment: Assessing Task Utility in LLM-Powered Applications

arXiv.org Artificial Intelligence

The rapid development in the field of Large Language Models (LLMs) has led to a surge in applications that facilitate collaboration among multiple agents to assist humans in their daily tasks. However, a significant gap remains in assessing whether LLM-powered applications genuinely enhance user experience and task execution efficiency. This highlights the pressing need for methods to verify utility of LLM-powered applications, particularly by ensuring alignment between the application's functionality and end-user needs. We introduce AgentEval provides an implementation for the math problems, a novel framework designed to simplify the utility verification process by automatically proposing a set of criteria tailored to the unique purpose of any given application. This allows for a comprehensive assessment, quantifying the utility of an application against the suggested criteria. We present a comprehensive analysis of the robustness of quantifier's work.


AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation

arXiv.org Artificial Intelligence

AutoGen is an open-source framework that allows developers to build LLM applications via multiple agents that can converse with each other to accomplish tasks. AutoGen agents are customizable, conversable, and can operate in various modes that employ combinations of LLMs, human inputs, and tools. Using AutoGen, developers can also flexibly define agent interaction behaviors. Both natural language and computer code can be used to program flexible conversation patterns for different applications. AutoGen serves as a generic infrastructure to build diverse applications of various complexities and LLM capacities. Empirical studies demonstrate the effectiveness of the framework in many example applications, with domains ranging from mathematics, coding, question answering, operations research, online decision-making, entertainment, etc.