Review for NeurIPS paper: Walsh-Hadamard Variational Inference for Bayesian Deep Learning

Walsh-Hadamard factorizations for variational posteriors are proposed. While reviewers appreciated the paper, the discussion brought to light several concerns shared across the reviewers. R1 in particular has updated their review to reflect some of these points from the discussion. It seems the proposed approaches are only applicable to a fully-connected last layer. There was a sense in the discussion that the authors had dodged these questions rather than addressing them directly. Last layer methods are certainly useful, and widely used in practice, such that this (significant) constraint would certainly be acceptable, if directly and honestly presented, alongside comparisons to such methods, such as the references [1,2] provided by R1.