Posterior Regularisation on Bayesian Hierarchical Mixture Clustering

The framework is founded on an approach of minimising the Kullback-Leibler (KL) divergence between a variational solution and the posterior, in a constrained space. The works (Dudík et al., 2004, 2007; Altun and Smola, 2006) first raised the idea of including constraints in maximum entropy density estimation and provided a theoretical analysis. Based on convex duality theory, the optimal solution of the regularised posterior is found to be the original posterior of the model, discounted by the constrained pseudo likelihood introduced by the constraints. Later work founded on the idea of posterior constraints includes (Graça et al., 2009) which proposed constraining the E-step of an Expectation-maximization (EM) algorithm, in order to impose feature constraints on the solution.