On Controllable Sparse Alternatives to Softmax

Anirban Laha, Saneem Ahmed Chemmengath, Priyanka Agrawal, Mitesh Khapra, Karthik Sankaranarayanan, Harish G. Ramaswamy

Neural Information Processing Systems 

Even though softmax is the most prevalent approach amongst them, it has a shortcoming in that its outputs are composed of only non-zeroes and is therefore ill-suited for producing sparse probability distributions as output.

Similar Docs  Excel Report  more

TitleSimilaritySource
None found