Neural Network -- Implementing Backpropagation using the Chain Rule

A neuron is a container that contains a mathematical function which is known as an activation function, inputs (x1 and x2 here), a vector of weights(w1,w2 here) and a bias(b). A neuron first computes the weighted sum of the inputs. The activation function is simply a mathematical function that takes in an input and produces an output. Think of the activation function as a mathematical operation that normalizes the input and produces an output. The output is then passed forward onto the neurons on the subsequent layer.