Understanding Deep Neural Networks from First Principles: Logistic Regression

The advanced feats we've seen machines do thus far have basically been examples of clever optimization techniques). So what does this learning process look like? First, weight and bias values are propagated forward through the model to arrive at a predicted output. At each neuron/node, the linear combination of the inputs is then multiplied by an activation function as described above-- the sigmoid function in our example. This process by which weights and biases are propagated from inputs to output is called forward propagation. After arriving at the predicted output, the loss for the training example is calculated.