Loss Functions in Deep Learning

For a more in-depth explanation of Forward Propagation and Backpropagation in neural networks, please refer to my other article What is Deep Learning and How does it work? For a given input vector x the neural network predicts an output, which is generally called a prediction vector y. We must compute a dot-product between the input vector x and the weight matrix W1 that connects the first layers with the second. After that, we apply a non-linear activation function to the result of the dot-product. Depending on the task we want the network to do, this prediction vector represents different things.