We present a method for automatically constructing macro-actions from scratch from primitive actions during the reinforcement learning process. The overall idea is to reinforce the tendency to perform action b after action a if such a pattern of actions has been rewarded. We test the method on a bicycle task, the car-on-the-hill task, the racetrack task and some grid-world tasks. For the bicycle and racetrack tasks the use of macro-actions approximately halves the learning time, while for one of the grid-world tasks the learning time is reduced by a factor of 5. The method did not work for the car-on-the-hill task for reasons we discuss in the conclusion. 1 INTRODUCTION A macro-action is a sequence of actions chosen from the primitive actions of the problem.

We have previously presented a coarse-to-fine hierarchical pyramid/neural network(HPNN) architecture which combines multiscale image processing techniques with neural networks.

One of the most important problems in visual perception is that of visual invariance: howare objects perceived to be the same despite undergoing transformations such as translations, rotations or scaling? In this paper, we describe a Bayesian method for learning invariances based on Lie group theory. We show that previous approaches based on first-order Taylor series expansions of inputs can be regarded as special cases of the Lie group approach, the latter being capable ofhandling in principle arbitrarily large transfonnations. Using a matrixexponential basedgenerative model of images, we derive an unsupervised algorithm for learning Lie group operators from input data containing infinitesimal transfonnations.

In contrast to gradient descentand EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthennore, the new parameter estimation procedure can be applied in both online and batch settings. We show experimentally that it is typically fasterthan EM, and usually requires about half as many iterations as EM. 1 Introduction Mixture models, in particular mixtures of Gaussians, have been a popular tool for density estimation, clustering, and unsupervised learning with a wide range of applications (see for instance [5, 2] and the references therein). Mixture models are one of the most useful tools for handling incomplete data, in particular hidden variables. For Gaussian mixtures the hidden variables indicate for each data point the index of the Gaussian that generated it.

We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in two different scenarios: (1) complex, twohanded actionrecognition in the martial art of Tai Chi and (2) detection and recognition of individual human behaviors and multiple-person interactions in a visual surveillance task. In the latter case, the system is particularly concerned with detecting when interactions between people occur, and classifying them. Graphical models, such as Hidden Markov Models (HMMs) [6] and Coupled Hidden MarkovModels (CHMMs) [3, 2], seem appropriate for modeling and, classifying human behaviors because they offer dynamic time warping, a well-understood training algorithm, and a clear Bayesian semantics for both individual (HMMs) and interacting or coupled (CHMMs) generative processes. A major problem with this data-driven statistical approach, especially when modeling rare or anomalous behaviors, is the limited number of training examples. A major emphasis of our work, therefore, is on efficient Bayesian integration of both prior knowledge with evidence from data.

We describe a unifying method for proving relative loss bounds for online linearthreshold classification algorithms, such as the Perceptron and the Winnow algorithms. For classification problems the discrete loss is used, i.e., the total number of prediction mistakes. We introduce a continuous lossfunction, called the "linear hinge loss", that can be employed to derive the updates of the algorithms. We first prove bounds w.r.t. the linear hinge loss and then convert them to the discrete loss. We introduce anotion of "average margin" of a set of examples . We show how relative loss bounds based on the linear hinge loss can be converted to relative loss bounds i.t.o. the discrete loss using the average margin.

A new algorithm for Support Vector regression is described.

The Workshop on Logic-Based Artificial Intelligence (LBAI) was held in Washington, D.C., on 13 to 15 June 1999. The workshop was organized by Jack Minker and John McCarthy. Its purpose was to bring together researchers who use logic as a fundamental tool in AI to permit them to review accomplishments, assess future directions, and share their research in LBAI.

Some application domains highlight the importance of distributed continual planning concepts; coordinating teams of unmanned ground vehicles in dynamic environments is an example of such a domain. In this article, I illustrate the ideas in, and promises of, distributed continual planning by showing how acquiring and distributing operator intent among multiple semiautonomous vehicles supports ongoing, cooperative mission elaboration and revision.

There's More to Life Than Making Plans: Plan Management in Dynamic, Multiagent Environments