Problem Solving
On the Evaluation of Engineering Artificial General Intelligence
Neema, Sandeep, Jha, Susmit, Nagel, Adam, Lew, Ethan, Sureshkumar, Chandrasekar, Gordic, Aleksa, Shimmin, Chase, Nguygen, Hieu, Eremenko, Paul
W e discuss the challenges and propose a framework for evalua ting engineering artificial general intelligence ( eAGI) agents. W e consider eAGI as a specialization of artificial general intelligence (AGI), deemed capab le of addressing a broad range of problems in the engineering of physical systems and associated controllers. W e exclude software engineering for a tractable s coping of eAGI and expect dedicated software engineering AI agents to address the software implementation challenges. Similar to human engineers, eAGI agents should possess a unique blend of background knowledge (recall and retrieve) of facts and methods, demonstrate familiarity with tools and processes, exhibit deep understanding of industrial components and well-known design families, and be able to engage in creative problem solving (analyze and synthesize), transf erring ideas acquired in one context to another . Given this broad mandate, evaluatin g and qualifying the performance of eAGI agents is a challenge in itself and, arguably, a critical ena bler to developing eAGI agents. In this paper, we address this challenge by proposin g an extensible evaluation framework that specializes and gr ounds Bloom's taxonomy - a framework for evaluating human learning that has also been recently used for evaluating LLMs - in an engineering design context. Our p roposed framework advances the state of the art in benchmarking and evaluation of AI agents in terms of the following: (a) developing a rich taxonomy of evaluati on questions spanning from methodological knowledge to real-world design proble ms; (b) motivating a pluggable evaluation framework that can evaluate not only t extual responses but also evaluate structured design artifacts such as CAD model s and SysML models; and (c) outlining an automatable procedure to customize the evaluation benchmark to different engineering contexts.
The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
Lee, Seongyun, Kim, Seungone, Seo, Minju, Jo, Yongrae, Go, Dongyoung, Hwang, Hyeonbin, Park, Jinho, Yue, Xiang, Welleck, Sean, Neubig, Graham, Lee, Moontae, Seo, Minjoon
Long chain-of-thought (CoT) is an essential ingredient in effective usage of modern large language models, but our understanding of the reasoning strategies underlying these capabilities remains limited. While some prior works have attempted to categorize CoTs using predefined strategy types, such approaches are constrained by human intuition and fail to capture the full diversity of model behaviors. In this work, we introduce the CoT Encyclopedia, a bottom-up framework for analyzing and steering model reasoning. Our method automatically extracts diverse reasoning criteria from model-generated CoTs, embeds them into a semantic space, clusters them into representative categories, and derives contrastive rubrics to interpret reasoning behavior. Human evaluations show that this framework produces more interpretable and comprehensive analyses than existing methods. Moreover, we demonstrate that this understanding enables performance gains: we can predict which strategy a model is likely to use and guide it toward more effective alternatives. Finally, we provide practical insights, such as that training data format (e.g., free-form vs. multiple-choice) has a far greater impact on reasoning behavior than data domain, underscoring the importance of format-aware model design.
Demystifying AI Agents: The Final Generation of Intelligence
McNamara, Kevin J, Marpu, Rhea Pritham
The trajectory of artificial intelligence (AI) has been one of relentless acceleration, evolving from rudimentary rule-based systems to sophisticated, autonomous agents capable of complex reasoning and interaction. This whitepaper chronicles this remarkable journey, charting the key technological milestones--advancements in prompting, training methodologies, hardware capabilities, and architectural innovations--that have converged to create the AI agents of today. We argue that these agents, exemplified by systems like OpenAI's ChatGPT with plugins and xAI's Grok, represent a culminating phase in AI development, potentially constituting the "final generation" of intelligence as we currently conceive it. We explore the capabilities and underlying technologies of these agents, grounded in practical examples, while also examining the profound societal implications and the unprecedented pace of progress that suggests intelligence is now doubling approximately every six months. The paper concludes by underscoring the critical need for wisdom and foresight in navigating the opportunities and challenges presented by this powerful new era of intelligence.
FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation
Guo, Jun, Ma, Xiaojian, Wang, Yikai, Yang, Min, Liu, Huaping, Li, Qing
This paper investigates training better visual world models for robot manipulation, i.e., models that can predict future visual observations by conditioning on past frames and robot actions. Specifically, we consider world models that operate on RGB-D frames (RGB-D world models). As opposed to canonical approaches that handle dynamics prediction mostly implicitly and reconcile it with visual rendering in a single model, we introduce FlowDreamer, which adopts 3D scene flow as explicit motion representations. FlowDreamer first predicts 3D scene flow from past frame and action conditions with a U-Net, and then a diffusion model will predict the future frame utilizing the scene flow. FlowDreamer is trained end-to-end despite its modularized nature. We conduct experiments on 4 different benchmarks, covering both video prediction and visual planning tasks. The results demonstrate that FlowDreamer achieves better performance compared to other baseline RGB-D world models by 7% on semantic similarity, 11% on pixel quality, and 6% on success rate in various robot manipulation domains.
EnerVerse-AC: Envisioning Embodied Environments with Action Condition
Jiang, Yuxin, Chen, Shengcong, Huang, Siyuan, Chen, Liliang, Zhou, Pengfei, Liao, Yue, He, Xindong, Liu, Chiming, Li, Hongsheng, Yao, Maoqing, Ren, Guanghui
Robotic imitation learning has advanced from solving static tasks to addressing dynamic interaction scenarios, but testing and evaluation remain costly and challenging due to the need for real-time interaction with dynamic environments. We propose EnerVerse-AC (EVAC), an action-conditional world model that generates future visual observations based on an agent's predicted actions, enabling realistic and controllable robotic inference. Building on prior architectures, EVAC introduces a multi-level action-conditioning mechanism and ray map encoding for dynamic multi-view image generation while expanding training data with diverse failure trajectories to improve generalization. As both a data engine and evaluator, EVAC augments human-collected trajectories into diverse datasets and generates realistic, action-conditioned video observations for policy testing, eliminating the need for physical robots or complex simulations. This approach significantly reduces costs while maintaining high fidelity in robotic manipulation evaluation. Extensive experiments validate the effectiveness of our method. Code, checkpoints, and datasets can be found at
A New Tractable Description Logic under Categorical Semantics
Duc, Chan Le, Brieulle, Ludovic
Biomedical ontologies contain numerous concept or role names involving negative knowledge such as lacks_part, absence_of. Such a representation with labels rather than logical constructors would not allow a reasoner to interpret lacks_part as a kind of negation of has_part. It is known that adding negation to the tractable Description Logic (DL) EL allowing for conjunction, existential restriction and concept inclusion makes it intractable since the obtained logic includes implicitly disjunction and universal restriction which interact with other constructors. In this paper, we propose a new extension of EL with a weakened negation allowing to represent negative knowledge while retaining tractability. To this end, we introduce categorical semantics of all logical constructors of the DL SH including EL with disjunction, negation, universal restriction, role inclusion and transitive roles. The categorical semantics of a logical constructor is usually described as a set of categorical properties referring to several objects without using set membership. To restore tractability, we have to weaken semantics of disjunction and universal restriction by identifying \emph{independent} categorical properties that are responsible for intractability, and dropping them from the set of categorical properties. We show that the logic resulting from weakening semantics is more expressive than EL with the bottom concept, transitive roles and role inclusion.
SparseMeXT Unlocking the Potential of Sparse Representations for HD Map Construction
Jiang, Anqing, Chai, Jinhao, Gao, Yu, Wang, Yiru, Heng, Yuwen, Sun, Zhigang, Sun, Hao, Zhao, Zezhong, Sun, Li, Zhou, Jian, Zhu, Lijuan, Xu, Shugong, Zhao, Hao
Recent advancements in high-definition \emph{HD} map construction have demonstrated the effectiveness of dense representations, which heavily rely on computationally intensive bird's-eye view \emph{BEV} features. While sparse representations offer a more efficient alternative by avoiding dense BEV processing, existing methods often lag behind due to the lack of tailored designs. These limitations have hindered the competitiveness of sparse representations in online HD map construction. In this work, we systematically revisit and enhance sparse representation techniques, identifying key architectural and algorithmic improvements that bridge the gap with--and ultimately surpass--dense approaches. We introduce a dedicated network architecture optimized for sparse map feature extraction, a sparse-dense segmentation auxiliary task to better leverage geometric and semantic cues, and a denoising module guided by physical priors to refine predictions. Through these enhancements, our method achieves state-of-the-art performance on the nuScenes dataset, significantly advancing HD map construction and centerline detection. Specifically, SparseMeXt-Tiny reaches a mean average precision \emph{mAP} of 55.5% at 32 frames per second \emph{fps}, while SparseMeXt-Base attains 65.2% mAP. Scaling the backbone and decoder further, SparseMeXt-Large achieves an mAP of 68.9% at over 20 fps, establishing a new benchmark for sparse representations in HD map construction. These results underscore the untapped potential of sparse methods, challenging the conventional reliance on dense representations and redefining efficiency-performance trade-offs in the field.
A Social Robot with Inner Speech for Dietary Guidance
Belcamino, Valerio, Carfรฌ, Alessandro, Seidita, Valeria, Mastrogiovanni, Fulvio, Chella, Antonio
We explore the use of inner speech as a mechanism to enhance transparency and trust in social robots for dietary advice. In humans, inner speech structures thought processes and decision-making; in robotics, it improves explainability by making reasoning explicit. This is crucial in healthcare scenarios, where trust in robotic assistants depends on both accurate recommendations and human-like dialogue, which make interactions more natural and engaging. Building on this, we developed a social robot that provides dietary advice, and we provided the architecture with inner speech capabilities to validate user input, refine reasoning, and generate clear justifications. The system integrates large language models for natural language understanding and a knowledge graph for structured dietary information. By making decisions more transparent, our approach strengthens trust and improves human-robot interaction in healthcare. We validated this by measuring the computational efficiency of our architecture and conducting a small user study, which assessed the reliability of inner speech in explaining the robot's behavior.
Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligence
Jiang, Jinhao, Chen, Changlin, Feng, Shile, Geng, Wanru, Zhou, Zesheng, Wang, Ni, Li, Shuai, Cui, Feng-Qi, Dong, Erbao
The ultimate goal of artificial intelligence (AI) is to achieve Artificial General Intelligence (AGI). Embodied Artificial Intelligence (EAI), which involves intelligent systems with physical presence and real-time interaction with the environment, has emerged as a key research direction in pursuit of AGI. While advancements in deep learning, reinforcement learning, large-scale language models, and multimodal technologies have significantly contributed to the progress of EAI, most existing reviews focus on specific technologies or applications. A systematic overview, particularly one that explores the direct connection between EAI and AGI, remains scarce. This paper examines EAI as a foundational approach to AGI, systematically analyzing its four core modules: perception, intelligent decision-making, action, and feedback. We provide a detailed discussion of how each module contributes to the six core principles of AGI. Additionally, we discuss future trends, challenges, and research directions in EAI, emphasizing its potential as a cornerstone for AGI development. Our findings suggest that EAI's integration of dynamic learning and real-world interaction is essential for bridging the gap between narrow AI and AGI.
PyResBugs: A Dataset of Residual Python Bugs for Natural Language-Driven Fault Injection
Cotroneo, Domenico, De Rosa, Giuseppe, Liguori, Pietro
It mentions modifying the put method and altering the release mechanism, leading to potential issues such as deadlocks or inconsistent states but avoids specifying exact code lines. This level provides testers with a broader understanding of the fault's behavior and consequences. In the High-Level Description (bottom right), we make the description entirely abstract and omit technical or contextual details about the specific fault. Modifying the put method introduces a " wrong algorithm small sparse modifications fault " in the fault-free function. This description suits scenarios where a conceptual understanding of the fault type is sufficient without providing implementation specifics. A team of six researchers specialized in computer engineering and cybersecurity created and validated the fault descriptions, under the coordination of a full professor with extensive expertise in software testing and fault injection. The professor established the description style, while the postdoctoral researcher, with a PhD in information technologies and background in AI and fault injection, provided ongoing reviews and feedback. The team, which also included a PhD student in cybersecurity and four M.Sc.