My Notes on Deep Learning Activation Functions.

The softmax activation is something like the combination of the multiple sigmoid functions. It is used to ensure that the total sum of the output from the layer is equal to 100%. For example, in multi-class classification problems where every observation belongs to exactly one class. So, here is the difference with Sigmoid: the first one allows the inputs/observations to belong to many classes. This activation may be placed in the final layer of the convolutional neural network (CNN) to classify the classes of hand-written digits (from 0 to 9).