Logistic Regression

We use different equations depending on the number of output classes. With 2 classes, we will use binomial cross entropy for loss and more than 2 classes involves using the cross-entropy with softmax. Example of how to calculate the cross-entropy loss for a 3 class problem. With the multinomial cross entropy, you can see that we only keep the loss contribution from the correct class. Usually, with neural nets, this will be case if our ouputs are sparse (just 1 true class).