LS-SVR as a Bayesian RBF network

Statistical learning theory has been studied for general function estimation from data since the late 1960's [22]. However, it was only widely adopted in practice after the introduction of the learning algorithms known as Support Vector Machines (SVMs) [23]. Using the so-called kernel trick, which replaces dot products between features and model parameters by evaluations of a kernel function, SVMs can learn nonlinear relations from training patterns by solving a convex optimization problem [16]. An important variant of the SVM is the Least Squares Support Vector Machine (LS-SVM) [20], which is obtained by making all data points supportvectors. LS-SVM avoids the constrained quadratic optimization step of standard SVMs by replacing the training procedure with one that reduces to solving a system of linear equations, which can be performed via ordinary least squares. The first SVM formulation was derived for classification tasks, but it has been readily adapted to tackle regression problems, being usually named Support Vector Regression (SVR) [6]. Similarly, the regression counterpart of LS-SVM is the LS-SVR [20]. 1