Markov Models
Links Between Markov Models and Multilayer Perceptrons
Bourlard, Hervé, Wellekens, C. J.
The use of a one-stage dynamic programming algorithm for connected word recognition.
The Boltzmann Perceptron Network: A Multi-Layered Feed-Forward Network Equivalent to the Boltzmann Machine
The concept of the stochastic Boltzmann machine (BM) is auractive for decision making and pattern classification purposes since the probability of attaining the network states is a function of the network energy. Hence, the probability of attaining particular energy minima may be associated with the probabilities of making certain decisions (or classifications). However, because of its stochastic nature, the complexity of the BM is fairly high and therefore such networks are not very likely to be used in practice. In this paper we suggest a way to alleviate this drawback by converting the stochastic BMinto a deterministic network which we call the Boltzmann Perceptron Network(BPN). The BPN is functionally equivalent to the BM but has a feed-forward structure and low complexity.
Optimization by Mean Field Annealing
Bilbro, Griff, Mann, Reinhold, Miller, Thomas K., Snyder, Wesley E., Bout, David E. van den, White, Mark
Nearly optimal solutions to many combinatorial problems can be found using stochastic simulated annealing. This paper extends the concept of simulated annealing from its original formulation as a Markov process to a new formulation based on mean field theory. Mean field annealing essentially replaces the discrete degrees offreedom in simulated annealing with their average values as computed by the mean field approximation. The net result is that equilibrium at a given temperature is achieved 1-2 orders of magnitude faster than with simulated annealing. A general framework forthe mean field annealing algorithm is derived, and its relationship toHopfield networks is shown. The behavior of MFA is examined both analytically and experimentally for a generic combinatorial optimizationproblem: graph bipartitioning. This analysis indicates the presence of critical temperatures which could be important inimproving the performance of neural networks.
Multilayer control of large Markov chains
A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
The optimal control of partially observable Markov processes over a finite horizon
Smallwood, R. D. | Sondik, E. J.
Operations Research, 21, 1071–1088.
The Optimal Control of Partially Observable Markov Decision Processes
Ph.D. thesis, Stanford University.
Statistical inference for probabilistic functions of finite state Markov chains
Annals of Mathematical Statistics, 41.
Optimal control of Markov decision processes with incomplete state estimation
J. Math. Anal. Applic., 10, 174–205.
Optimal control of partially observable Markov systems
This paper discusses the discrete-time Bayesian optimal control of stochastic dynamic systems where some vectors, which augment the system state vectors and the observed state vectors by additional variables, constitute multi-dimensional Markov chains. Optimal control of such Markovian control systems is considered under the assumption that only a part of the components of such vectors is observed by the control system. Certain conditional probability densities needed in deriving optimal control policies are derived, and computational procedures which determine optimal control sequences are given.