Sparse Algorithm for Robust LSSVM in Primal Space

Li Chen a,b, Shuisheng Zhou a, a School of Mathematics and Statistics, Xidian University, 266 Xinglong Section, Xifeng Road, Xi'an, China b Department of Basic Science, College of Information and Business, Zhongyuan Technology University, 41 Zhongyuan Middle Road, Zhengzhou, ChinaAbstract As enjoying the closed form solution, least squares support vector machine (LSSVM) has been widely used for classification and regression problems having the comparable performance with other types of SVMs. However, LSSVM has two drawbacks: sensitive to outliers and lacking sparseness. Robust LSSVM (R-LSSVM) overcomes the first partly via nonconvex truncated loss function, but the current algorithms for R-LSSVM with the dense solution are faced with the second drawback and are inefficient for training large-scale problems. In this paper, we interpret the robustness of R-LSSVM from a re-weighted viewpoint and give a primal R-LSSVM by the representer theorem. The new model may have sparse solution if the corresponding kernel matrix has low rank. Then approximating the kernel matrix by a low-rank matrix and smoothing the loss function by entropy penalty function, we propose a convergent sparse R-LSSVM (SR-LSSVM) algorithm to achieve the sparse solution of primal R-LSSVM, which overcomes two drawbacks of LSSVM simultaneously. The proposed algorithm has lower complexity than the existing algorithms and is very efficient for training large-scale problems. Many experimental results illustrate that SR-LSSVM can achieve better or comparable performance with less training time than related algorithms, especially for training large scale problems. Keywords: Primal LSSVM, Sparse solution, Re-weighted LSSVM, Low-rank approximation, Outliers 2010 MSC: 00-01, 99-00 1. Introduction Least squares support vector machine (LSSVM) was introduced by Suykens[1] and has been a powerful learning technique for classification and regression. It has been successfully used in many real world pattern recognition problems, such as disease diagnosis[2], fault detection[3], image classification [4], partial differential equations solving[5] and visual tracking[6]. LSSVM tries to minimize least squares errors on the training samples.