An Introduction to Neural Networks

My last note looked at the problem of overfitting machine learning models and a best practice to avoid it called regularization. This note looks at neural networks, what they are, and how they work conceptually. In its most simple form, a neural network can be just an input, a node (or neuron) where a calculation uses that input, and the output of the calculation. This by itself isn't very powerful, but when you combine many of these nodes into a network, they can operate like how neurons in our own brain work. Using linear algebra and calculus to execute a concept called backpropagation of error, these networks of neurons can automatically adjust the weights of the connections between them by assessing how prediction errors map back to different connections in the network.