One Simple Trick to Fix Your Bayesian Neural Network