Submodular Optimization for Efficient Semi-supervised Support Vector Machines

Abstract--In this work we present a quadratic programming approximation of the Semi-Supervised Support V ector Machine (S3VM) problem, namely approximate QP-S3VM, that can be efficiently solved using off the shelf optimization packages. We prove that this approximate formulation establishes a relation between the low density separation and the graph-based models of semi-supervised learning (SSL) which is important to develop a unifying framework for semi-supervised learning methods. Furthermore, we propose the novel idea of representing SSL problems as submodular set functions and use efficient sub-modular optimization algorithms to solve them. Using this new idea we develop a representation of the approximate QP-S3VM as a maximization of a submodular set function which makes it possible to optimize using efficient greedy algorithms. We demonstrate that the proposed methods are accurate and provide significant improvement in time complexity over the state of the art in the literature. The recent advances in information technology imposes serious challenges on traditional machine learning algorithms where classification models are trained using labeled samples. Data collection and storage nowadays has never been easier and therefore using such enormous volumes of data to infer reliable classification models is of utmost importance.