Supplement to " Structured Dropout Variational Inference for Bayesian Neural Networks "

In Appendix A, we analyze the expressiveness of V ariational Structured Dropout (VSD) through the approximate posterior structure and the parameterization of prior hierarchy. In Appendix B, we provide proof for the KL condition in VSD. In Appendix C, we derive in details the variational objective of VSD with hierarchical prior. A essential question is how expressive the Dropout posterior in VSD is. MC Dropout objective is a lower bound on the scale mixture model's marginal MAP objective.