Reviews: Learning towards Minimum Hyperspherical Energy

To reduce the redundancy in representation, this paper extends some recent work on diversity regularization in neural networks by proposing the so-called hyperspherical potential energy defined on the Euclidean distance or the angular distance, which, when minimized, helps to increase the diversity of the input weights of different neurons and hence reduce the redundancy. The regularized loss function contains two regularization terms corresponding to the hidden units and the output units, respectively.