SVD-Softmax: Fast Softmax Approximation on Large Vocabulary Neural Networks

We propose a fast approximation method of a softmax function with a very large vocabulary using singular value decomposition (SVD).