Hyperspherical Alternatives to Softmax

In the context of classification problems, a softmax classifier with a cross-entropy loss is often the go-to approach. However, in situations with many classes, softmax can be slow to train as it requires an output node for every class, leading to very large output layers. For example, a network with a hidden layer size of 300 and 100,000 output classes has 30 million parameters in the output layer alone. In applied AI settings, these types of problems occur often. An example of this is learning to match papers with authors, or product descriptions with actual products.