In this paper, we study the problem of learning the structure of a discrete set of N tokens based on their interactions with other tokens. We focus on a setting where the tokens can be partitioned into a small number of classes, and there exists a real-valued function f defined on certain sets of tokens. This function, which captures the interactions between tokens, depends only on the class memberships of its arguments. The goal is to recover the class memberships of all tokens from a finite number of samples of f . We begin by analyzing this problem from both complexity-theoretic and information-theoretic viewpoints.

Transformers have achieved great success in recent years. Interestingly, transformers have shown particularly strong in-context learning capability -- even without fine-tuning, they are still able to solve unseen tasks well purely based on task-specific prompts. In this paper, we study the capability of one-layer transformers in learning the one-nearest neighbor prediction rule. Under a theoretical framework where the prompt contains a sequence of labeled training data and unlabeled test data, we show that, although the loss function is nonconvex, when trained with gradient descent, a single softmax attention layer can successfully learn to behave like a one-nearest neighbor classifier. Our result gives a concrete example on how transformers can be trained to implement nonparametric machine learning algorithms, and sheds light on the role of softmax attention in transformer models.

Recent advancements in text-to-image (T2I) and vision-language-to-image (VL2I) generation have made significant strides. However, the generation from generalized vision-language inputs, especially involving multiple images, remains under-explored. This paper presents Kosmos-G, a model that leverages the advanced perception capabilities of Multimodal Large Language Models (MLLMs) to tackle the aforementioned challenge. Our approach aligns the output space of MLLM with CLIP using the textual modality as an anchor and performs compositional instruction tuning on curated data. Kosmos-G demonstrates a unique capability of zero-shot multi-entity subject-driven generation. Notably, the score distillation instruction tuning requires no modifications to the image decoder. This allows for a seamless substitution of CLIP and effortless integration with a myriad of U-Net techniques ranging from fine-grained controls to personalized image decoder variants. We posit Kosmos-G as an initial attempt towards the goal of "image as a foreign language in image generation."

Machines can now do what you could call IQ-style thinking – covering what'multiple intelligences' theorist Howard Gardner would call visual-spatial, verbal-linguistic and logical-mathematical intelligence – pretty darn well. Artificial Intelligence (AI) is here and it's getting more sophisticated every day. But AC – Artificial Creativity – barely exists. AI has been unsettling the human world for quite some time. Can you believe it was 1997 when IBM's Deep Blue computer triumphed over chess colossus Garry Kasparov?

There's More to Life Than Making Plans For many years, research in AI plan generation was governed by a number of strong, simplifying assumptions: The planning agent is omniscient, its actions are deterministic and instantaneous, its goals are fixed and categorical, and its environment is static. More recently, researchers have developed expanded planning algorithms that are not predicated on such assumptions, but changing the way in which plans are formed is only part of what is required when the classical assumptions are abandoned. The demands of dynamic, uncertain environments mean that in addition to being able to form plans--even probabilistic, uncertain plans--agents must be able to effectively manage their plans. In this article, which is based on a talk given at the 1998 AAAI Fall Symposium on Distributed, Continual Planning, we first identify reasoning tasks that are involved in plan management, including commitment management, environment monitoring, alternative assessment, plan elaboration, metalevel control, and coordination with other agents. We next survey approaches we have developed to many of these tasks and discuss a plan-management system we are building to ground our theoretical work, by providing us with a platform for integrating our techniques and exploring their value in a realistic problem.

The Fourth International and Interdisciplinary Conference on Modeling and Using Context (CONTEXT-03) took place at the Stanford University Center for the Study of Language and Information in Stanford, California, on 23 to 25 June 2003. Like the previous conferences, CONTEXT-03 fulfilled its aim of bringing together representatives of many different research areas, spanning the whole range of the cognitive and information sciences, and with interests ranging from the use of context in specific, commercial applications to highly general philosophical, psychological, and logical theories. The conference chair was Fausto Giunchiglia, University of Trento. The program chairs were Patrick Blackburn, INRIA Lorraine; Chiara Ghidini, the Centre for Scientific and Technological Research in Trento; and Roy Turner, University of Maine. There were 77 submissions, from which 31 papers and 14 posters were selected. One of the aims of the CONTEXT conferences is to bring together representatives of ...

Johanna Moore's work in the area of computer-generated explanation has been highly influential. Her thesis work, as well as the subsequent work of her and her students, has helped to change the way we think about the problem of generating explanations. The crux of the explanation problem, according to Moore, is not how to present information as such but how to impart an understanding on the user. The explanation system should be flexible enough that if an initial explanation fails to convey the understanding, it can try explaining the concept in a different way. The system should be aware of what it previously said to the user and what its communicative goals were at the time.

This workshop brought together 20 computer scientists, psychologists, and human-computer interaction (HCI) researchers to exchange results and views on human error and judgment bias. Human error is typically studied when operators undertake actions, but judgment bias is an issue in thinking rather than acting. Both topics are generally ignored by the HCI community, which is interested in designs that eliminate human error and bias tendencies. As a result, almost no one at the workshop had met before, and the discussion for most participants was novel and lively. Many areas of previously unexamined overlap were identified.

The importance of contextual reasoning is emphasized by various researchers in AI. (A partial list includes John McCarthy and his group, R. V. Guha, Yoav Shoham, Giuseppe Attardi and Maria Simi, and Fausto Giunchiglia and his group.) Here, we survey the problem of formalizing context and explore what is needed for an acceptable account of this abstract notion. Although the word context is frequently used in descriptions, explanations, and analyses of computer programs in these areas, its meaning is frequently left to the reader's understanding; that is, it is used in an implicit and intuitive manner. An example of how contexts may help in AI is found in McCarthy's (constructive) criticism (McCarthy 1984) of I wish honorable gentlemen would have the fairness to give the entire context of what I did say, and not pick out detached words (R. Cobden [1849], quoted in Oxford English Dictionary [1978], p. 902). The main motivation for studying formal contexts is to resolve the problem of generality in AI.