In contrast to k-nearest neighbors, a simple example of a parametric method would be logistic regression, a generalized linear model with a fixed number of model parameters: a weight coefficient for each feature variable in the dataset plus a bias (or intercept) unit. While the learning algorithm optimizes an objective function on the training set (with exception to lazy learners), hyperparameter optimization is yet another task on top of it; here, we typically want to optimize a performance metric such as classification accuracy or the area under a Receiver Operating Characteristic curve. Thinking back of our discussion about learning curves and pessimistic biases in Part II, we noted that a machine learning algorithm often benefits from more labeled data; the smaller the dataset, the higher the pessimistic bias and the variance -- the sensitivity of our model towards the way we partition the data. We start by splitting our dataset into three parts, a training set for model fitting, a validation set for model selection, and a test set for the final evaluation of the selected model.
Scientists have used deep learning algorithms with multiple processing layers (hence "deep") to make better models from large quantities of unlabeled data (such as photos with no description, voice recordings or videos on YouTube). Google's voice recognition algorithms operate with a massive training set -- yet it's not nearly big enough to predict every possible word or phrase or question you could put to it. And Google's deep learning algorithm discovers cats. Algorithms perform superior face recognition tasks using deep network that take into account 120 million parameters.
More importantly, however, Google and its competitors are moving towards keying their search algorithms to understand natural speech as well, in anticipation of more and more voice search. But new machine learning algorithms are making more accurate, real-time translations possible. You might also be interested in my new big data case study collection, which you can download for free from here: Big Data Case Study Collection: 7 Amazing Companies That Really Get Big Data. My current book is Big Data: Using Smart Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance' and my new books (available to pre-order now) are Key Business Analytics: The 60 Business Analysis Tools Every Manager Needs To Know and Big Data in Practice.
Last week, machine learning took a big leap forward when Google's AlphaGo, a machine algorithm, beat the world champion, Lee Sedol, in the game Go. When IBM Watson beat former champions Ken Jennings and Brad Rutter in the game show Jeopardy! Even though it doesn't rely on encoded rules, IBM Watson requires close monitoring by domain experts to provide data and evaluate its performance. AlphaGo was programmed to seek positive rewards in the form of scores and continually improve its system by playing millions of games against tweaked versions of itself.