Kinks In the Works

Note that for each neuron in our network, we were able to define another hinge point, or "kink" in the line, that got added to the function so that the slope of the prediction line would change based on the sum of all of the neurons. In this instance, we had two neurons, and because their activation points were different (x 5 and x 14), this resulted in two kinks at those points. While this is a significantly more complicated prediction function than a standard linear classifier could produce, it still might not be complex enough. If we wanted to get another kink in our predictions, we could accomplish this by simply adding a third neuron. For each new neuron in our network, we're able to add another kink to our prediction line, with the slope at each kink changing based on the activation point (determined by the ReLU activation function.)