Bayesian Learning
Maximum Likelihood Competitive Learning
One popular class of unsupervised algorithms are competitive algorithms. Inthe traditional view of competition, only one competitor, the winner, adapts for any given case. I propose to view competitive adaptationas attempting to fit a blend of simple probability generators (such as gaussians) to a set of data-points. The maximum likelihoodfit of a model of this type suggests a "softer" form of competition, in which all competitors adapt in proportion to the relative probability that the input came from each competitor. I investigate one application of the soft competitive model, placement ofradial basis function centers for function interpolation, and show that the soft model can give better performance with little additional computational cost. 1 INTRODUCTION Interest in unsupervised learning has increased recently due to the application of more sophisticated mathematical tools (Linsker, 1988; Plumbley and Fallside, 1988; Sanger, 1989) and the success of several elegant simulations of large scale selforganization (Linsker,1986; Kohonen, 1982). One popular class of unsupervised algorithms are competitive algorithms, which have appeared as components in a variety of systems (Von der Malsburg, 1973; Fukushima, 1975; Grossberg, 1978). Generalizing the definition of Rumelhart and Zipser (1986), a competitive adaptive system consists of a collection of modules which are structurally identical except, possibly, for random initial parameter variation.
Bayesian Inference of Regular Grammar and Markov Source Models
Smith, Kurt R., Miller, Michael I.
In this paper we develop a Bayes criterion which includes the Rissanen complexity, for inferring regular grammar models. We develop two methods for regular grammar Bayesian inference. The fIrst method is based on treating the regular grammar as a I-dimensional Markov source, and the second is based on the combinatoric characteristics of the regular grammar itself. We apply the resulting Bayes criteria to a particular example in order to show the efficiency of each method.
Identifying independence in Bayesian networks
Geiger, D., Verma, T., Pearl, J.
An important feature of Bayesian networks is that they facilitate explicit encoding of information about independencies in the domain, information that is indispensable for efficient inferencing. This article characterizes all independence assertions that logically follow from the topology of a network and develops a linear time algorithm that identifies these assertions. The algorithm's correctness is based on the soundness of a graphical criterion, called d-separation, and its optimality stems from the completeness of d-separation. An enhanced version of d-separation, called D-separation, is defined, extending the algorithm to networks that encode functional dependencies. Finally, the algorithm is shown to work for a broad class of nonprobabilistic independencies.
Symbolic probabilistic inference in belief networks
Shachter, R. D., Ambrosio, B., Favero, B. A.
Dรญez's algorithm for the noisy MAX is very efficient for polytrees, but when the network has loops, it has to be combined with local conditioning, a suboptimal propagation algorithm. Other algorithms, based on several factorizations of the conditional probability of the noisy MAX, are not as efficient for polytrees but can be combined with general propagation algorithms such as clustering or variable elimination, which are more efficient for networks with loops. In this article we propose a new factorization of the noisy MAX that amounts to Dรญez's algorithm in the case of polytrees and at the same time is more efficient than previous factorizations when combined with either variable elimination or clustering.
The computational complexity of probabilistic inference using Bayesian belief networks
Bayesian belief networks provide a natural, efficient method for representing probabilistic dependencies among a set of variables. For these reasons, numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence. Algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks. More general classes of belief networks, however, have eluded efforts to develop efficient inference algorithms. We show that probabilistic inference using belief networks is NP-hard.
Influence Diagrams, Belief Nets and Decision Analysis
Oliver, R. M. | Smith, J. Q. (Eds.)
Based on the proceedings of a conference on Influence Diagrams for Decision Analysis, Inference and Prediction held at the University of California at Berkeley in May of 1988, this is the first book devoted to the subject. The editors have brought together recent results from researchers actively investigating influence diagrams and also from practitioners who have used influence diagrams in developing models for problem-solving in a wide range of fields.
Performance of Synthetic Neural Network Classification of Noisy Radar Signals
Ahalt, Stanley C., Garber, F. D., Jouny, I., Krishnamurthy, Ashok K.
This study evaluates the performance of the multilayer-perceptron and the frequency-sensitive competitive learning network in identifying five commercial aircraft from radar backscatter measurements. The performance of the neural network classifiers is compared with that of the nearest-neighbor and maximum-likelihood classifiers. Our results indicate that for this problem, the neural network classifiers are relatively insensitive to changes in the network topology, and to the noise level in the training data. While, for this problem, the traditional algorithms outperform these simple neural classifiers, we feel that neural networks show the potential for improved performance.