The First International Workshop on User Modeling in Natural Language Dialogue Systems was held 30-31 August 1986 in Maria Laach, West Germany Issues addressed by the participants included the appropriate contents of a user model, techniques for constructing user models, strategies for reasoning on user models in both understanding and generating natural language dialogue, and the development of general user-modeling systems This article includes an overview of the presentations made at the workshop It is a compilation of the author's impressions and observations and is, therefore, undoubtedly incomplete; and at times might fail to accurately represent the views of the researcher presenting the work The workshop was organized by Dr. Wolfgang Wahlster and Dr. Alfred Kobsa, both of the University of Saarbriicken, and was supported by a grant from the German Science Foundation in its Special Collaborative Program on AI and Knowledge-Based Systems. Twenty-four invited researchers from seven countries participated in the workshop. The program included both long and short talks on current research ideas and projects and lively discussion among the participants; oftentimes, the participants became so engrossed in the presentations and ensuing discussions that other aspects of the program, including the banquet, had to be delayed. But all agreed the workshop had been an enjoyable experience and extremely worthwhile. The workshop program included talks on a wide spectrum of topics related to user modeling in natural language dialogue systems.

Editors' Note: In this provocative article Doyle suggests that I thank Jaime Carbonell, John McDermott, Joseph Schatz, and Derek Sleeman for helpful discussions and comments This research was supported by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 3597, monitored by the Air Force Avionics Laboratory under Contract F33615-81-K-1539. The views and conclusions contained in this document are those of the author, and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Government of the United States of America Abstract, Knowledge engineers qualified to build expert systems are currently in short supply The production of useful and trustworthy cxpcrt systems can he significantly increased by pursuing the idea of nrCiculate ayprentzce.ship This revolution is very important. We now actively seek out tasks for automation that would never have been considered previously. It seems clear that the work of our society and industry includes many economically important (if often mundane) tasks whose automation may be possible with the new techniques.

Early research on AI typically involved qualitative demonstrations of intelligent behavior, with novelty being the primary focus. However, as the field has matured, there have been increasing demands for more careful evaluation using quantitative measures of behavior. In some cases, the response has taken the guise of formal analyses, and in others, it has emphasized comparisons between system and human behavior, but the predominant movement has been toward empirical studies of AI methods. As a result, techniques for experimental design, exploratory data analysis, and statistical testing, originally developed in other fields, have become increasingly relevant for AI researchers. Paul Cohen's book Empirical Methods for Artificial Intelligence aims to encourage this trend by providing AI practitioners with the knowledge and tools needed for careful empirical evaluation.

In the last few years, we have witnessed a major growth in the use of empirical methods in AI. In part, this growth has arisen from the availability of fast networked computers that allow certain problems of a practical size to be tackled for the first time. There is also a growing realization that results obtained empirically are no less valuable than theoretical results. Experiments can, for example, offer solutions to problems that have defeated a theoretical attack and provide insights that are not possible from a purely theoretical analysis. I identify some of the emerging trends in this area by describing a recent workshop that brought together researchers using empirical methods as far apart as robotics and knowledge-based systems.

We provide a brief overview of recent direct preference elicitation methods: these methods ask users to answer (ideally, a small number of) queries regarding their preferences and use this information to recommend a feasible decision that would be (approximately) optimal given those preferences. We argue for the importance of assessing numerical utilities rather than qualitative preferences and survey several utility elicitation techniques from artificial intelligence, operations research, and conjoint analysis. Specifically, since the ability to make reasonable decisions on behalf of a user depends on that user's preferences over outcomes in the domain in question, AI systems must assess or estimate these preferences before making decisions. Designing effective preference assessment techniques to incorporate such user-specific considerations (that is, breaking the preference bottleneck) is one of the most important problems facing AI. In this brief survey, we focus on explicit elicitation techniques where a system actively queries a user to glean relevant preferences. Preference elicitation is difficult for two main reasons. First, many decision problems have exponentially sized outcome spaces, defined by the possible values of outcome attributes. As an illustrative example, consider sophisticated flight selection: possible outcomes are defined by attributes such as trip cost, departure time, return time, airline, number of connections, flight length, baggage weight limit, flight class, (the possibility of) lost luggage, flight delays, and other stochastic outcomes. An ideal decision support system should be able to use, for example, precise flight delay statistics and incorporate a user's relative tolerance for delays in making recommendations. Representing and eliciting preferences for all outcomes in a case like this is infeasible given the size of the outcome space. A second difficulty arises due to the fact that quantitative strength of preferences, or utility, is needed to trade off, for instance, the odds of flight delays with other attributes. Unfortunately, people are notoriously inept at quantifying their preferences with any degree of precision, adding to the challenges facing automated utility elicitation.

"An Innovative Application from the DARPA Knowledge Bases Programs: Rapid Development of a Course-of-Action Critiquer," by Gheorghe Tecuci, Mihai Boicu, Mike Bowman, and Dorin Marcu, describes a critiquing agent for military courses of action, a challenge problem set by the Defense Advanced Research Projects Agency's (DARPA) High-Performance Knowledge Bases Program. Murray Burke, the DARPA manager for this program, introduces the article by setting the context for the application. Ontologies also play a key role in the creation and management of a web portal developed by Steffen Staab and his colleagues at the University of Karlsruhe, discussed in their article, "Knowledge Portals: Ontologies at Work." "L As in past years, papers were solicited in two categories: (1) deployed applications and (2) emerging applications and technologies. Deployed applications are systems that have been in use for at least several months by individuals or organizations other than their developers, have measurable benefits, and incorporate AI technologies. Emerging applications are systems that are close to deployment and clearly show an innovative implementation of AI technologies. Papers submitted in this track can also describe efforts that examine the utility of different AI techniques for specific applications. All these case studies are of value not only to other application developers looking for guidance in applying various techniques to their own applications but also to researchers who need to understand the technical challenges provided by real-world problems. Six deployed applications and 12 emerging application papers were presented plus 2 invited talks. Although no single theme emerges from this panoply of excellent applications, they served to demonstrate that the field continues to be fertile ground for innovation.

Ramasamy Uthurusamy was the program chair, and Barbara Hayes-Roth was the program cochair. IAAI-99 was a special occasion that provided an opportunity to reflect on a decade of IAAI conferences and contemplate the potential contributions in the coming decade. In addition to the three invited talks that specifically addressed these areas, IAAI-99 showcased some exciting and innovative applications. Although all the IAAI-99 papers and talks were certainly interesting and important, we present in this special issue of AI Magazine only a select subset because of page and other limitations. We include two invited talks and four applications as a snapshot of IAAI-99.

DITORIAL AI Magazine Volume 11 Number 2 (1990) ( AAAI) In this issue, Luc Steels takes a new and insightful look at knowledgebased systems and provides a synthesis of several different approaches to analyzing expertise. It's a long article but, in my opinion, an important one. I recommend it to anyone with an interest in knowledge-level analysis of expert systems. On the same general topic of expert systems but from a different perspective is the article by Rob Weitz, who proposes a methodology for forecasting the impact of expert systems on the workplace over the near term. Finally, James Hendler, Austin Tate, and Mark Drummond present an extensive survey of AI systems and techniques for plan generation.

A variety of proposals for preferred methodological approaches has been advanced in the recent artificial intelligence (AI) literature Rather than advocating a particular approach, this article attempts to explain the apparent confusion of efforts in the field in terms of differences among underlying methodological perspectives held by practicing researchers The article presents a review of such perspectives discussed in the existing literature and then considers a descriptive and relatively specific typology of these differing research perspectives. Studies are reported in a wide range of publications. While some focus on the field (e.g., Artzficial Intelligence), others are concerned with different research areas (e.g., Behavzoral and Brain Sczences). Perhaps, as others have pointed out, "there are undoubtedly some views AI simply adds to the prevailing sense of confusion. AI research, which have been previously reported in .

This article provides an overview of the state of the art in this area. It begins with the dominant paradigm of assistive control, from impedance-based cooperative controller through electromyography and intention estimation. It then covers challenge-based algorithms, which provide more difficult and complex tasks for the patient to perform through resistive control and error augmentation. Furthermore, it describes exercise adaptation algorithms that change the overall exercise intensity based on the patient's performance or physiological responses, as well as socially assistive robots that provide only verbal and visual guidance. The article concludes with a discussion of the current challenges in rehabilitation robot software: evaluating existing control strategies in a clinical setting as well as increasing the robot's autonomy using entirely new artificial intelligence techniques.