Reviews: Learning Invariances using the Marginal Likelihood

In this manuscript the authors present a scheme for ranking and refining invariance transformations within the Bayesian approach to supervised learning, thereby offering a fully Bayesian alternative to data augmentation. The implementation presented also introduces a scheme for unbiased estimation of the evidence lower bound for transformation representation models having a Normal likelihood (or writeable in an equivalent form as per the Polya-Gamma example). Moreover, the authors have been careful to ensure that their implementation is structured for efficiency within the contemporary framework for stochastic optimisation of sparse approximations to the GP (variational inference with mini-batching, etc.). Although limited by space, the examples presented are convincing of the potential of this approach. It is my view that this is a valuable and substantial contribution to the field, although I would be prepared to concede in relation to the NIPS reviewer guidelines that in some sense the progress is incremental rather than revolutionary.