Conventional control-theoretic models for fault detection typically rely on an accu(cid:173) rate model ofthe plant being monitored (Patton, Frank, and Clark, 1989). However, in practice it common that no such model exists for complex non-linear systems. The large ground antennas used by JPL's Deep Space Network (DSN) to track

We introduce a new approach for on-line recognition of handwrit(cid:173) ten words written in unconstrained mixed style. The preprocessor performs a word-level normalization by fitting a model of the word structure using the EM algorithm. Words are then coded into low resolution "annotated images" where each pixel contains informa(cid:173) tion about trajectory direction and curvature. The recognizer is a convolution network which can be spatially replicated. From the network output, a hidden Markov model produces word scores.

We propose a learning algorithm for a variable memory length Markov process. Human communication, whether given as text, handwriting, or speech, has multi characteristic time scales. On short scales it is characterized mostly by the dynamics that gen(cid:173) erate the process, whereas on large scales, more syntactic and se(cid:173) mantic information is carried. For that reason the conventionally used fixed memory Markov models cannot capture effectively the complexity of such structures. On the other hand using long mem(cid:173) ory models uniformly is not practical even for as short memory as four. The algorithm we propose is based on minimizing the sta(cid:173) tistical prediction error by extending the memory, or state length, adaptively, until the total prediction error is sufficiently small.

If data collection is costly, there is much to be gained by actively se(cid:173) lecting particularly informative data points in a sequential way. In a Bayesian decision-theoretic framework we develop a query selec(cid:173) tion criterion which explicitly takes into account the intended use of the model predictions. By Markov Chain Monte Carlo methods the necessary quantities can be approximated to a desired preci(cid:173) sion. As the number of data points grows, the model complexity is modified by a Bayesian model selection strategy. The proper(cid:173) ties of two versions of the criterion ate demonstrated in numerical experiments.

We introduce a recurrent architecture having a modular structure and we formulate a training procedure based on the EM algorithm. The resulting model has similarities to hidden Markov models, but supports recurrent networks processing style and allows to exploit the supervised learning paradigm while using maximum likelihood estimation.

This paper presents ongoing work on a speaker independent visual speech recognition system. The work presented here builds on previous research efforts in this area and explores the potential use of simple hidden Markov models for limited vocabulary, speaker independent visual speech recognition. The task at hand is recognition of the first four English digits, a task with possible applications in car-phone images were modeled as mixtures of independent dialing. The Gaussian distributions, and the temporal dependencies were captured with standard left-to-right hidden Markov models. The results indicate that simple hidden Markov models may be used to successfully recognize relatively unprocessed image sequences.

Most of the common techniques for estimating conditional prob(cid:173) ability densities are inappropriate for applications involving peri(cid:173) odic variables. In this paper we introduce three novel techniques for tackling such problems, and investigate their performance us(cid:173) ing synthetic data. We then apply these techniques to the problem of extracting the distribution of wind vector directions from radar scatterometer data gathered by a remote-sensing satellite.

We propose a statistical mechanical framework for the modeling of discrete time series. Maximum likelihood estimation is done via Boltzmann learning in one-dimensional networks with tied weights.

This paper studies the problem of diffusion in Markovian models, such as hidden Markov models (HMMs) and how it makes very difficult the task of learning of long-term dependencies in sequences. Using results from Markov chain theory, we show that the problem of diffusion is reduced if the transition probabilities approach 0 or 1. Under this condition, standard HMMs have very limited modeling capabilities, but input/output HMMs can still perform interesting computations.

Multi-class classification problems can be efficiently solved by partitioning the original problem into sub-problems involving only two classes: for each pair of classes, a (potentially small) neural network is trained using only the data of these two classes. We show how to combine the outputs of the two-class neural networks in order to obtain posterior probabilities for the class decisions. The resulting probabilistic pairwise classifier is part of a handwriting recognition system which is currently applied to check reading. We present results on real world data bases and show that, from a practical point of view, these results compare favorably to other neural network approaches.