On the benefits of non-linear weight updates