As a teacher of Data Science (Data Science for Internet of Things course at the University of Oxford), I am always fascinated in cross connection between concepts. To recap, Logistic regression is a binary classification method. It can be modelled as a function that can take in any number of inputs and constrain the output to be between 0 and 1. This means, we can think of Logistic Regression as a one-layer neural network. I hope you found this analysis useful as well.
This is part 2/3 of a series on deep learning and deep belief networks. This section will focus on artificial neural networks (ANNs) by building upon the logistic regression model we learned about last time. It'll be a little shorter because we already built the foundation for some very important topics in part 1 – namely the objective / error function and gradient descent. We will focus on 2 main functions of ANNs – the forward pass (prediction) and backpropagation (learning). Your sci-kit learn analogues would be model.predict()
The advanced feats we've seen machines do thus far have basically been examples of clever optimization techniques). So what does this learning process look like? First, weight and bias values are propagated forward through the model to arrive at a predicted output. At each neuron/node, the linear combination of the inputs is then multiplied by an activation function as described above-- the sigmoid function in our example. This process by which weights and biases are propagated from inputs to output is called forward propagation. After arriving at the predicted output, the loss for the training example is calculated.
Neural networks are a set of algorithms, modeled loosely after the human brain, that are designed to recognize patterns. They interpret sensory data through a kind of machine perception, labeling or clustering raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text or time series, must be translated. Neural networks help us cluster and classify. You can think of them as a clustering and classification layer on top of the data you store and manage. They help to group unlabeled data according to similarities among the example inputs, and they classify data when they have a labeled dataset to train on. What kind of problems does deep learning solve, and more importantly, can it solve yours?