How do we decide how to represent an intelligent system in its interface, and how do we decide how the interface represents information about the world and about its own workings to a user? This article addresses these questions by examining the interaction between representation and intelligence in user interfaces. The rubric representation covers at least three topics in this context: (1) how a computational system is represented in its user interface, (2) how the interface conveys its representations of information and the world to human users, and (3) how the system's internal representation affects the human user's interaction with the system. I argue that each of these kinds of representation (of the system, information and the world, the interaction) is key to how users make the kind of attributions of intelligence that facilitate their interactions with intelligent systems. In this vein, it makes sense to represent a systmem as a human in those cases where social collaborative behavior is key and for the system to represent its knowledge to humans in multiple ways on multiple modalities. I demonstrate these claims by discussing issues of representation and intelligence in an embodied conversational agent -- an interface in which the system is represented as a person, information is conveyed to human users by multiple modalities such as voice and hand gestures, and the internal representation is modality independent and both propositional and nonpropositional.

Choosing an environment is an important decision for agent developers. A key issue in this decision is whether the environment will provide realistic problems for the agent to solve, in the sense that the problems are true to the issues that arise in addressing a particular research question. In addition to realism, other important issues include how tractable problems are that can be formulated in the environment, how easy agent performance can be measured, and whether the environment can be customized or extended for specific research questions. In the ideal environment, researchers can pose realistic but tractable problems to an agent, measure and evaluate its performance, and iteratively rework the environment to explore increasingly ambitious questions, all at a reasonable cost in time and effort. As might be expected, trade-offs dominate the suitability of an environment; however, we have found that the modern graphic user interface offers a good balance among these trade-offs. This article takes a brief tour of agent research in the user interface, showing how significant questions related to vision, planning, learning, cognition, and communication are currently being addressed.

We describe an approach to intelligent user interfaces, based on the idea of making the computer a collaborator, and an application-independent technology for implementing such interfaces.

The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency n_ij/n leading to a point estimate I(n_ij/n) for the mutual information. To answer questions like "is I(n_ij/n) consistent with zero?" or "what is the probability that the true mutual information is much larger than the point estimate?" one has to go beyond the point estimate. In the Bayesian framework one can answer these questions by utilizing a (second order) prior distribution p(t) comprising prior information about t. From the prior p(t) one can compute the posterior p(t|n), from which the distribution p(I|n) of the mutual information can be calculated. We derive reliable and quickly computable approximations for p(I|n). We concentrate on the mean, variance, skewness, and kurtosis, and non-informative priors. For the mean we also give an exact expression. Numerical issues and the range of validity are discussed.

In this paper, we present algorithms that perform gradient ascent of the average reward in a partially observable Markov decision process (POMDP). These algorithms are based on GPOMDP, an algorithm introduced in a companion paper (Baxter & Bartlett, this volume), which computes biased estimates of the performance gradient in POMDPs. The algorithm's chief advantages are that it uses only one free parameter beta, which has a natural interpretation in terms of bias-variance trade-off, it requires no knowledge of the underlying state, and it can be applied to infinite state, control and observation spaces. We show how the gradient estimates produced by GPOMDP can be used to perform gradient ascent, both with a traditional stochastic-gradient algorithm, and with an algorithm based on conjugate-gradients that utilizes gradient information to bracket maxima in line searches. Experimental results are presented illustrating both the theoretical results of (Baxter & Bartlett, this volume) on a toy problem, and practical aspects of the algorithms on a number of more realistic problems.

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes POMDPs controlled by parameterized stochastic policies. A similar algorithm was proposed by (Kimura et al. 1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free beta (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter beta is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter et al., this volume) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward.

We briefly describe the implementation and evaluation of a novel plan synthesis system, called AltAlt. AltAlt is designed to exploit the complementary strengths of two of the currently popular competing approaches for plan generation: (1) graphplan and (2) heuristic state search. It uses the planning graph to derive effective heuristics that are then used to guide heuristic state search. The heuristics derived from the planning graph do a better job of taking the subgoal interactions into account and, as such, are significantly more effective than existing heuristics. AltAlt was implemented on top of two state-of-the-art planning systems: (1) stan3.0, a graphplan-style planner, and (2) hsp-r, a heuristic search planner.

We briefly describe the implementation and evaluation of a novel plan synthesis system, called AltAlt. AltAlt is designed to exploit the complementary strengths of two of the currently popular competing approaches for plan generation: (1) graphplan and (2) heuristic state search. It uses the planning graph to derive effective heuristics that are then used to guide heuristic state search. The heuristics derived from the planning graph do a better job of taking the subgoal interactions into account and, as such, are significantly more effective than existing heuristics. AltAlt was implemented on top of two state-of-the-art planning systems: (1) stan3.0, a graphplan-style planner, and (2) hsp-r, a heuristic search planner.

That is, it does not do Fikes and Nilsson (1971) (called forwardchaining, something like "to achieve g, try to achieve g This factor seemed to be crucial in making [1998]). Briefly speaking, given a conjunction our algorithm effective on some of of simple goals, it selects one of them to work domains at the competition. In general, some of goal based on the actions that have this these conditions can contain variables, and simple goal as one of their effects and recursively these variables are eventually grounded by tries to achieve the new goal. If there is no user-provided information, on starting up is the initial state description. If all In comparison, our algorithm does not of them are true already, then it performs the modify the initial state description. The planner then selects an action maintains a global list Γ that represents the that can make g true; consider first the conjunction sequence of actions that the planner has found of all ground preconditions of this to this point.

For more details, see Nau et al. 's preconditions can include logical inferences, 's preconditions two methods for traveling from one location can include Horn-clause inferencing, numeric to another: (1) traveling by airplane and (2) computations, and calls to external programs. 's expressive power can be used to create a totally ordered list of subtasks. Suppose domain representations for complex application that all these subtasks are primitive except for domains. For example, the Horn 4. if t is primitive (i.e., there is an operator for t) then clauses can include calls to attached procedures 5. nondeterministically choose an operator o for t We believe the primary 14. endif's higher level of expressivity made it possible to formulate highly expressive domain algorithms in's data structures to make them faster; for example, we found that a simple change to the data structure We intend to make more optimizations in the near future. (Aha and Breslow 1997).