Reviews: Learning Neural Networks with Adaptive Regularization

Statistical Strength Throughoutt the paper, you refer to the concept of'statistical strength' without describing what it actually means. I expect it means that if two things are correlated, you can estimate properties of them with better sample efficiency if you take this correlation into account, since you're effectively getting more data. Given that two features are correlated, optimization will be improved if you do some sort of preconditioning that accounts for this structure. In other words, given that features are correlated, you want to'share statistical strength.' However, it is less clear to me why you want to regularize the model such that things become correlated/anti-correlated.