Reviews: From Stochastic Planning to Marginal MAP

Main ideas The paper develops the relation between solving an MDP and performing inference in a Bayesian network. The direction, however, is novel as far as I can tell: using MDP algorithms to solve an inference problem. The first part shows that an existing MDP algorithm (ARollout) is in fact performing a BP iteration over the DBN that represents the MDP. In the second part, a different MDP algorithm (SOGBOFA) is used to solve a particular inference problem of choosing a subset of values with the maximal marginals (MMAP). The resulting SOGBOFA-based solver often loses to the state-of-the-art, but for harder cases it can outperform the state of the art.