Learning Neural Networks with Adaptive Regularization

Specifically,we propose to construct amatrix-variate normal prior (on weights) whose covariance matrix hasaKroneckerproduct structure. Thisstructure is designed to capture the correlations in neurons through backpropagation.