Replica Analysis of the Linear Model with Markov or Hidden Markov Signal Priors

This paper estimates free energy, average mutual information, and minimum mean square error (MMSE) of a linear model under two assumptions: (1) the source is generated by a Markov chain, (2) the source is generated via a hidden Markov model. Our estimates are based on the replica method in statistical physics. We show that under the posterior mean estimator, the linear model with Markov sources or hidden Markov sources is decoupled into single-input AWGN channels with state information available at both encoder and decoder where the state distribution follows the left Perron-Frobenius eigenvector with unit Manhattan norm of the stochastic matrix of Markov chains. Numerical results show that the MMSEs obtained via the replica method are good lower bounds to the mean square errors (MSEs) achieved by some well-known approximate message passing algorithms in the research literature. Our estimates are based on the replica method which was developed originally to study mean field approximations in spin glasses [1]. Although this method lacks of rigorous mathematical proof in some particular parts, it has been widely accepted as an analytic tool and utilized to investigate a variety of problems in applied mathematics, information processing, and coding [2]. L. V Truong is with the Department of Engineering, University of Cambridge. A. Related Work The use of the replica method for studying multiuser estimators goes back to [3] where Tanaka determined the asymptotic bit error rate of Marginal-Posterior-Mode (MPM) estimators by employing the replica method.