Our framework, HARMONIC (Human-AI Robotic Team Member Operating with Natural Intelligence and Communication, Figure 1), is an implemented dual-control cognitive robotic architecture featuring distinct layers of strategic reasoning and tactical, skill-level control [20]. This approach advances the hybrid control systems and architectures reviewed by Dennis et al. [21] and contrasts with DIARC's [22], [23] integration strategy, which embeds the strategic layer within the tactical layer to support concurrent operation. The strategic layer of HARMONIC adapts a mature cognitive architecture, OntoAgent [24], [25], [17] for high-level reasoning, leveraging explicit, structured knowledge representations that can be inspected, verified, and incre-mentally expanded.

Shapes of cognition is a new conceptual paradigm for the computational cognitive modeling of Language - Endowed Intelligent Agents (LEIAs) . S hapes are remembered constellations of sensory, linguistic, conceptual, episodic, and procedural knowledge that allow agents to cut through the complexity of real life the same way as people do: by expecting things to be typical, recognizing patterns, acting by habit, reasoning by analogy, satisficing, and generally minimizing cognitive load to the degree situations permit . Atypical outcomes are treated using shapes - based recovery method s, such as learning on the fly, asking a human partner for help, or seeking an actionable, even if imperfect, situational understanding . Although shapes is an umbrella term, it is not vague: shapes - based modeling involves particular objectives, hypotheses, modeling strategies, knowledge bases, and actual models of wide - ranging phenomena, all implemented within a particular cognitive architecture . Such s pecificity is needed both to vet the our hypotheses and to achieve our practical aims of building useful agent systems that are explainable, extensible, and worthy of our trust, even in critical domains . However, a lthough the LEIA example of shapes - based modeling is specific, the principles can be applied more broadly, giving new life to knowledge - based and hybrid AI .

For AI agents to emulate human behavior, they must be able to perceive, meaningfully interpret, store, and use large amounts of information about the world, themselves, and other agents. Metacognition is a necessary component of all of these processes. In this paper, we briefly a) introduce content-centric computational cognitive (C4) modeling for next-generation AI agents; b) review the long history of developing C4 agents at RPI's LEIA (Language-Endowed Intelligent Agents) Lab; c) discuss our current work on extending LEIAs' cognitive capabilities to cognitive robotic applications developed using a neuro symbolic processing model; and d) sketch plans for future developments in this paradigm that aim to overcome underappreciated limitations of currently popular, LLM-driven methods in AI.

Explanation is key to people having confidence in high-stakes AI systems. However, machine-learning-based systems -- which account for almost all current AI -- can't explain because they are usually black boxes. The explainable AI (XAI) movement hedges this problem by redefining "explanation". The human-centered explainable AI (HCXAI) movement identifies the explanation-oriented needs of users but can't fulfill them because of its commitment to machine learning. In order to achieve the kinds of explanations needed by real people operating in critical domains, we must rethink how to approach AI. We describe a hybrid approach to developing cognitive agents that uses a knowledge-based infrastructure supplemented by data obtained through machine learning when applicable. These agents will serve as assistants to humans who will bear ultimate responsibility for the decisions and actions of the human-robot team. We illustrate the explanatory potential of such agents using the under-the-hood panels of a demonstration system in which a team of simulated robots collaborate on a search task assigned by a human.

We present HARMONIC, a framework for implementing cognitive robots that transforms general-purpose robots into trusted teammates capable of complex decision-making, natural communication and human-level explanation. The framework supports interoperability between a strategic (cognitive) layer for high-level decision-making and a tactical (robot) layer for low-level control and execution. We describe the core features of the framework and our initial implementation, in which HARMONIC was deployed on a simulated UGV and drone involved in a multi-robot search and retrieval task.

The paper describes a system that uses large language model (LLM) technology to support the automatic learning of new entries in an intelligent agent's semantic lexicon. The process is bootstrapped by an existing non-toy lexicon and a natural language generator that converts formal, ontologically-grounded representations of meaning into natural language sentences. The learning method involves a sequence of LLM requests and includes an automatic quality control step. To date, this learning method has been applied to learning multiword expressions whose meanings are equivalent to those of transitive verbs in the agent's lexicon. The experiment demonstrates the benefits of a hybrid learning architecture that integrates knowledge-based methods and resources with both traditional data analytics and LLMs.

This paper describes principles and practices of knowledge engineering that enable the development of holistic language-endowed intelligent agents that can function across domains and applications, as well as expand their ontological and lexical knowledge through lifelong learning. For illustration, we focus on dialog act modeling, a task that has been widely pursued in linguistics, cognitive modeling, and statistical natural language processing. We describe an integrative approach grounded in the OntoAgent knowledge-centric cognitive architecture and highlight the limitations of past approaches that isolate dialog from other agent functionalities.

Developing cognitive agents with human-level natural language understanding (NLU) capabilities requires modeling human cognition because natural, unedited utterances regularly contain ambiguities, ellipses, production errors, implicatures, and many other types of complexities. Moreover, cognitive agents must be nimble in the face of incomplete interpretations since even people do not perfectly understand every aspect of every utterance they hear. So, once an agent has reached the best interpretation it can, it must determine how to proceed – be that acting upon the new information directly, remembering an incomplete interpretation and waiting to see what happens next, seeking out information to fill in the blanks, or asking its interlocutor for clarification. The reasoning needed to support NLU extends far beyond language itself, including, non-exhaustively, the agent’s understanding of its own plans and goals; its dynamic modeling of its interlocutor’s knowledge, plans, and goals, all guided by a theory of mind; its recognition of diverse aspects human behavior, such as affect, cooperative behavior, and the effects of cognitive biases; and its integration of linguistic interpretations with its interpretations of other perceptive inputs, such as simulated vision and non-linguistic audition. Considering all of these needs, it seems hardly possible that fundamental NLU will ever be achieved through the kinds of knowledge-lean text-string manipulation being pursued by the mainstream natural language processing (NLP) community. Instead, it requires a holistic approach to cognitive modeling of the type we are pursuing in a paradigm called OntoAgent.

This paper details how OntoAgents, language-endowed intelligent agents developed in the OntoAgent framework, assess their confidence in understanding language inputs. It presents scoring heuristics for the following subtasks of natural language understanding: lexical disambiguation and the establishment of semantic dependencies; reference resolution; nominal compounding; the treatment of fragments; and the interpretation of indirect speech acts. The scoring of confidence in individual linguistic subtasks is a prerequisite for computing the overall confidence in the understanding of an utterance. This, in turn, is a prerequisite for the agent’s deciding how to act upon that level of understanding.

Ellipsis is a linguistic process that makes certain aspects of text meaning not directly traceable to surface text elements and, therefore, inaccessible to most language processing technologies. However, detecting and resolving ellipsis is an indispensable capability for language-enabled intelligent agents. The key insight of the work presented here is that not all cases of ellipsis are equally difficult: some can be detected and resolved with high confidence even before we are able to build agents with full human-level semantic and pragmatic understanding of text. This paper describes a fully automatic, implemented and evaluated method of treating one class of ellipsis: elided scopes of modality. Our cognitively-inspired approach, which centrally leverages linguistic principles, has also been applied to overt referring expressions with equally promising results.