Goto

Collaborating Authors

 Industry


Experiments with Infinite-Horizon, Policy-Gradient Estimation

Journal of Artificial Intelligence Research

In this paper, we present algorithms that perform gradient ascent of the average reward in a partially observable Markov decision process (POMDP). These algorithms are based on GPOMDP, an algorithm introduced in a companion paper (Baxter & Bartlett, this volume), which computes biased estimates of the performance gradient in POMDPs. The algorithm's chief advantages are that it uses only one free parameter beta, which has a natural interpretation in terms of bias-variance trade-off, it requires no knowledge of the underlying state, and it can be applied to infinite state, control and observation spaces. We show how the gradient estimates produced by GPOMDP can be used to perform gradient ascent, both with a traditional stochastic-gradient algorithm, and with an algorithm based on conjugate-gradients that utilizes gradient information to bracket maxima in line searches. Experimental results are presented illustrating both the theoretical results of (Baxter & Bartlett, this volume) on a toy problem, and practical aspects of the algorithms on a number of more realistic problems.


RIACS Workshop on the Verification and Validation of Autonomous and Adaptive Systems

AI Magazine

The long-term future of space exploration at the National Aeronautics and Space Administration (NASA) is dependent on the full exploitation of autonomous and adaptive systems, but mission managers are worried about the reliability of these more intelligent systems. The main focus of the workshop was to address these worries; hence, we invited NASA engineers working on autonomous and adaptive systems and researchers interested in the verification and validation of software systems. The dual purpose of the meeting was to (1) make NASA engineers aware of the verification and validation techniques they could be using and (2) make the verification and validation community aware of the complexity of the systems NASA is developing. The workshop was held 5 to 7 December 2000 at the Asilomar Conference Center in Pacific Grove, California.


A Gamut of Games

AI Magazine

In 1950, Claude Shannon published his seminal work on how to program a computer to play chess. In Shannon's time, it would have seemed unlikely that only a scant 50 years would be needed to develop programs that play world-class backgammon, checkers, chess, Othello, and Scrabble. Computer games research is one of the important success stories of AI. This article reviews the past successes, current projects, and future research directions for AI using computer games as a research test bed.


The Shop Planning System

AI Magazine

Shop is a hierarchical task network planning algorithm that is provably sound and complete across a large class of planning domains. It plans for tasks in the same order that they will later be executed, and thus, it knows the current world state at each step of the planning process. For example, shop's preconditions can include logical inferences, complex numeric computations, and calls to external programs.


AAAI 2001 Spring Symposium Series Reports

AI Magazine

The Association for the Advancement of Artificial Intelligence, in cooperation with Stanford University's Department of Computer Science, presented the 2001 Spring Symposium Series on Monday through Wednesday, 26 to 28 March 2001, at Stanford University. The titles of the seven symposia were (1) Answer Set Programming: Toward Efficient and Scalable Knowledge, Representation and Reasoning, (2) Artificial Intelligence and Interactive Entertainment, (3) Game-Theoretic and Decision-Theoretic Agents, (4) Learning Grounded Representations, (5) Model-Based Validation of Intelligence, (6) Robotics and Education, and (7) Robust Autonomy.


The Shop Planning System

AI Magazine

For more details, see Nau et al. 's preconditions can include logical inferences, 's preconditions two methods for traveling from one location can include Horn-clause inferencing, numeric to another: (1) traveling by airplane and (2) computations, and calls to external programs. 's expressive power can be used to create a totally ordered list of subtasks. Suppose domain representations for complex application that all these subtasks are primitive except for domains. For example, the Horn 4. if t is primitive (i.e., there is an operator for t) then clauses can include calls to attached procedures 5. nondeterministically choose an operator o for t We believe the primary 14. endif's higher level of expressivity made it possible to formulate highly expressive domain algorithms in's data structures to make them faster; for example, we found that a simple change to the data structure We intend to make more optimizations in the near future. (Aha and Breslow 1997).


AAAI News

AI Magazine

C. Furniture, Fixtures and Equipment: Effective for 1996 the Association Furniture, fixtures and equipment are has changed its method of accounting stated at cost, less accumulated depreciation.


AAAI 2001 Spring Symposium Series Reports

AI Magazine

The Association for the Advancement of Artificial Intelligence, in cooperation with Stanford University's Department of Computer Science, presented the 2001 Spring Symposium Series on Monday through Wednesday, 26 to 28 March 2001, at Stanford University. The titles of the seven symposia were (1) Answer Set Programming: Toward Efficient and Scalable Knowledge, Representation and Reasoning, (2) Artificial Intelligence and Interactive Entertainment, (3) Game-Theoretic and Decision-Theoretic Agents, (4) Learning Grounded Representations, (5) Model-Based Validation of Intelligence, (6) Robotics and Education, and (7) Robust Autonomy.


A Gamut of Games

AI Magazine

In Shannon's time, it would have seemed Around this time, Arthur Samuel began work the capabilities of computational intelligence. By 1958, Alan Newell and Herb Simon the game world with the real world--the game had begun their investigations into chess, of life--where the rules often change, the which eventually led to fundamental results scope of the problem is almost limitless, and for AI and cognitive science (Newell, Shaw, and the participants interact in an infinite number Simon 1958). An impressive lineup to say the of ways. Games can be a microcosm of the real least! Indeed, one of the early goals of AI was to and chess programs could play at a build a program capable of defeating the level comparable to the human world champion. This These remarkable accomplishments are the challenge proved to be more difficult than was result of a better understanding of the anticipated; the AI literature is replete with problems being solved, major algorithmic optimistic predictions. It eventually took insights, and tremendous advances in hardware almost 50 years to complete the task--a technology. The work on computer remarkably short time when one considers the games has been one of the most successful and software and hardware advances needed to visible results of AI research. The results are truly of the progress in building a world-class amazing. Even though there is an exponential program for the game is given, along with a difference between the best case and the brief description of the strongest program. The histories are necessarily case (Plaat et al. 1996). Games reports the past successes where computers realizing the lineage of the ideas.


Creativity at the Metalevel: AAAI-2000 Presidential Address

AI Magazine

Creativity is sometimes taken to be an inexplicable aspect of human activity. By summarizing a considerable body of literature on creativity, I hope to show how to turn some of the best ideas about creativity into programs that are demonstrably more creative than any we have seen to date. I believe the key to building more creative programs is to give them the ability to reflect on and modify their own frameworks and criteria. That is, I believe that the key to creativity is at the metalevel.