An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods

A learning problem is referred to as classification if its output take discrete values in a set of possible categories and regression if it has continuous real-valued output. A simple and useful model of an input-output functional relationship is to assume that the output variable can be expressed approximately as a linear combination of its input vector components. These linear models include the linear least squares method for regression and the logistic regression method for classification. Because a linear model has limited prediction power by itself, there has been extensive research in nonlinear models such as neural networks. However, there are two major problems with the use of nonlinear models: First, they are theoretically difficult to analyze, and second, they are computationally difficult to solve.