Computer Sczence Department Carnegae-Mellon University Pittsburgh, Pennsylvania 152125 Two CLOSELY RELATED ASPECTS of Artificial Intelligence t,hat have received comparatively little attention in the recent literature are research methodology and the analysis of computational techniques that span multiple application areas. We believe both issues to be increasingly significant, as Artificial Intelligence matures into a scienceand spins off major application efforts. It is imperative to analyze the repertoire of AI methods with respect to past experience, utility in new domains, extensibility, and functional equivalence with other techniques, if N is to become more effective in building upon prior results rather than continually reinverking the proverbial wheel. Similarly, awareness of research methodology issues can help plan future research by learning from past successes and failures. We view the study of research methodology to be similar to the analysis of operational AI techniques, but at a met,a-level; that is, research methodology analyzes the techniques and methods used by the researchers themselves, rather than their programs, to resolve issues of selecting interesting and tract,able problems to investigate, and of deciding how to proceed with t,heir investigations.

If the user is satisfied with the expected consequences, s/he finalizes his(her) decisions. The quality of his(her) decisions are automatically evaluated, and information about the strengths and shortcomings of his(her) decision-making strategy is fed back to him(her). We have used this environment to train and evaluate novice air traffic controllers. Figure 5 shows the two displays of the PMME system. The left-hand side displays the current world and the right-hand side the extrapolated world.

This survey was conducted for technical and historical reasons: First, I work in the commercial AI industry and was worried about missing significant intellectual contributions to my work. Second, this work was intended to test the thesis that there is a coherent body of study called cognitive science. If a new scientific discipline has emerged that is a fusion of psychology, computer science, linguistics, mathematics, philosophy, and neuroscience (as claimed by Gardner 1985), then there should be some evidence of this new discipline in the pattern of scientific publications and researcher biographies. For example, a paper about AI could cite a psychology paper, or a graduate in a mathematics department could migrate into the linguistics field. I have been informally conducting this survey for the last three years.

Deploying AI systems on the Web provides tangible evidence of the power and utility of AI techniques. Next time you encounter AI bashing, wouldn't it be satisfying to counter with a handful of well-chosen URLs? At the conference, Jude Shavlik asked me to edit a special issue of AI Magazine describing AI systems that have the Web as their domain. Indeed, the authors of each article included in this special issue have promised to create and maintain a URL pointing to a working prototype. Now, almost a year later, we have the fruit of this labor.

It has become especially relevant in the last decade, with researchers increasingly investigating how to identify the most suitable existing algorithm for solving a problem instance instead of developing new algorithms. This survey presents an overview of this work focusing on the contributions made in the area of combinatorial search problems, where algorithm selection techniques have achieved significant performance improvements. We unify and organise the vast literature according to criteria that determine algorithm selection systems in practice. The comprehensive classification of approaches identifies and analyzes the different directions from which algorithm selection has been approached. This article contrasts and compares different methods for solving the problem as well as ways of using these solutions.

AI's War on Manipulation: Are We Winning? The next day was going to be a big day: Citizens of Bitotia would once and for all establish which byte order was better, big-endian (B) or little-endian (L). Little Bit Timmy was a big supporter of little endian because that would give him the best position in the word. However, the population was split quite evenly between L and B, with a small minority of Bits who still remembered the single-tape Turing machine and preferred unary encoding (U), without any of this endianness business. Nonetheless, about half of the Bits preferred big-endian (B L U), and about half were the other way round (L B U).

In this article, we first survey the three major types of computer music systems based on AI techniques: (1) compositional, (2) improvisational, and (3) performance systems. Representative examples of each type are briefly described. Then, we look in more detail at the problem of endowing the resulting performances with the expressiveness that characterizes human-generated music. This is one of the most challenging aspects of computer music that has been addressed just recently. The main problem in modeling expressiveness is to grasp the performer's "touch," that is, the knowledge applied when performing a score.

This special issue of AI Magazine is dedicated to the proposition that problems populate the path to insight, implying that experiences and lessons learned should be shared. In this view, problems are signposts, not roadblocks, which guide us towards better solutions. Bugs, surprises, and anomalies also become powerful instructional tools that reveal assumptions, expose design flaws, and chart the boundaries of current technology. This perspective motivated our work on this special issue. When researchers publish success stories, we commonly leave process out of the narrative.

This special issue is based on the premise that in order to achieve human-level artificial intelligence researchers will have to find ways to integrate insights from multiple computational frameworks and to exploit insights from other fields that study intelligence. Articles in this issue describe recent approaches for integrating algorithms and data structures from diverse subfields of AI. Much of this work incorporates insights from neuroscience, social and cognitive psychology or linguistics. The new applications and significant improvements to existing applications this work has enabled demonstrates the ability of integrated systems and research to continue progress towards human-level artificial intelligence. However, we believe that progress towards human-level artificial intelligence and the applications it enables requires a deeper and more comprehensive understanding that cannot be achieved by studying individual areas in isolation.

The Seventeenth American Association of Artificial Intelligence (AAAI) and the Twelfth Innovative Applications of Artificial Intelligence (IAAI) conferences were jointly held in Austin, Texas, on 30 July through 3 August. They continue to provide a growing wealth of research stretching into many of the areas of AI. Coupled with demonstrations of the emerging and deployed IAAI techniques, the joint conference results in a complete survey of breaking technology. With the decade's events in highperformance desktops and "invisible" computing, AI has flourished and now expands from its classical cognitive architecture roots into intense gaming and wireless devices. AI successfully reaches these areas with its continuing multidisciplinary collaborations and creativity.