Multilayer Perceptrons (MLP), are complex algorithms that take a lot of compute power and a *ton* of data in order to produce satisfactory results in reasonable timeframes. Let's start with what they're not: neural networks, despite the name and every blog post and intro to machine learning text book you've probably read up till now, are not analogs of the human brain. There are some *very* surface-level similarities, but the actual functionality of a neural network has almost nothing in common with the neurons that make up the approximately three pounds of meat that sits between your ears and defines everything you do and how you experience reality. Just like a lot of other machine learning algorithms, they use the formula "label equals weight times data value plus offset" (or y w*x b) to define where they draw their lines/hyperplanes for making predictions. In machine learning, that slope is called a weight.)
But first let's get our bearings. What exactly is a neural network trying to do? Like any other model, it's trying to make a good prediction. We have a set of inputs and a set of target values -- and we are trying to get predictions that match those target values as closely as possible. Forget for a second the more complicated looking picture of the neural network I drew above and focus on this simpler one below.
Logistic regression is another technique borrowed by machine learning from the field of statistics. It is the go-to method for binary classification problems (problems with two class values). In this post you will discover the logistic regression algorithm for machine learning. This post was written for developers interested in applied machine learning, specifically predictive modeling. You do not need to have a background in linear algebra or statistics.
This post basically takes the tutorial on Classifying MNIST digits using Logistic Regression which is primarily written for Theano and attempts to port it to Keras. So, what better way to put that claim to the test than to write some code! Keras comes with great documentation. One can really get up and running in a matter of minutes. Everything needed to accomplish the goal can be found on the Guide to Sequential Model page (assuming of course the initial setup and configuration is all taken care of).
The question of which model type to apply to a Machine Learning task can be a daunting one given the immense number of algorithms available in the literature. It can be difficult to compare the relative merits of two methods, as one can outperform the other in a certain class of problems while consistently coming in behind for another class. In this post, the last one of our series of posts about Logistic Regression, we'll explore the differences between Decision Trees and Logistic Regression for classification problems, and try to highlight scenarios where one might be recommended over the other. Logistic Regression and trees differ in the way that they generate decision boundaries i.e. the lines that are drawn to separate different classes. To illustrate this difference, let's look at the results of the two model types on the following 2-class problem: Decision Trees bisect the space into smaller and smaller regions, whereas Logistic Regression fits a single line to divide the space exactly into two.