Multi-Modal Cognitive States: Augmenting the State in Cognitive Architectures

AAAI Conferences

Idealization of intelligence as an embodied activity, involving an integration of cognition, perception and the body, places the tightest constraints on the design space for AI artifacts, forcing AI to deeply understand the design tradeoffs and tricks that biology has developed. I propose that a step in the design of such artifacts is to broaden the notion of cognitive state from the current linguistic-symbolic, Language-of-Thought framework to a multi-modal one, where perception and kinesthetic modalities participate in thinking. This is in contrast to the roles assigned to perception and motor activities as modules external to central cognition in the currently dominant theories in AI and Cognitive Science. I develop the outlines of this proposal, and describe the implementation of a bimodal version in which a diagrammatic representation component is added to the cognitive state.


Multi-modal Systems As Multi-representational Systems

AAAI Conferences

In earlier work, we have shown how a cognitive architecture can be augmented with a diagrammatic reasoning system to produce a bimodal cognitive architecture. In this paper, we show how this bimodal architecture is also bi-representational (multi-representational in the general case) by describing a desiderata for representational formalisms and showing how the diagrammatic representation in biSoar satisfies these requirements.


A Diagrammatic Reasoning Architecture: Design, Implementation and Experiments

AAAI Conferences

This paper explores the idea that the cognitive state during problem solving diagrams is bimodal, one of whose components is the traditional predicate-symbolic representation composed of relations between entities in the domain of interest, while a second component is an internal diagrammatic representation. In parallel with the operators in the symbolic representation that are based on symbol matching and inferencing, there is a set of operators in the diagrammatic component that apply perceptions to the elements of the diagram to generate information. In addition there is a set of diagram construction operations that may modify the diagram by adding, deleting and modifying the diagrammatic elements, in the service of problem solving goals. We describe the design of the diagrammatic component of the architecture, and show how the symbolic and diagrammatic modes collaborate in the solution of a problem. We end the paper with a view of the cognitive state as multi-modal, in consonance with our own phenomenal sense of experiencing the world in multiple modalities and using these senses in solving problems.


A Bimodal Cognitive Architecture: Explorations in Architectural Explanation of Spatial Reasoning

AAAI Conferences

Research in Psychology often involves the building of computational models to test out various theories. The usual approach is to build models using the most convenient tool available. Newell has instead proposed building models within the framework of general-purpose cognitive architectures. One advantage of this approach is that in some cases it is possible to provide more perspicuous explanations of experimental results in different but related tasks, as emerging from an underlying architecture. In this paper, we propose the use of a bimodal cognitive architecture called biSoar in modeling phenomena in spatial representation and reasoning. We show biSoar can provide an architectural explanation for the phenomena of simplification that arises in experiments associated with spatial recall. We build a biSoar model for one such spatial recall task - wayfinding, and discuss the role of the architecture in the emergence of simplification.


Reasoning with Diagrammatic Representations

AI Magazine

We report on the spring 1992 symposium on diagrammatic representations in reasoning and problem solving sponsored by the American Association for Artificial Intelligence. The symposium brought together psychologists, computer scientists, and philosophers to discuss a range of issues covering both externally represented diagrams and mental images and both psychologyand AIrelated issues. In this article, we develop a framework for thinking about the issues that were the focus of the symposium as well as report on the discussions that took place. We anticipate that traditional symbolic representations will increasingly be combined with iconic representations in future AI research and technology and that this symposium is simply the first of many that will be devoted to this topic. The emphasis of this symposium was diagrammatic (or pictorial) representations in problem solving and reasoning.