Sigsoftmax: Reanalysis of the Softmax Bottleneck

Sekitoshi Kanai, Yasuhiro Fujiwara, Yuki Yamanaka, Shuichi Adachi

Neural Information Processing Systems 

Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning.