In this paper we establish a duality between boosting and SVM, and use this to derive a novel discriminant dimensionality reduction algorithm. In particular, using the multiclass formulation of boosting and SVM we note that both use a combination of mapping and linear classification to maximize the multiclass margin. In SVM this is implemented using a pre-defined mapping (induced by the kernel) and optimizing the linear classifiers. In boosting the linear classifiers are pre-defined and the mapping (predictor) is learned through combination of weak learners. We argue that the intermediate mapping, e.g.

Neural Information Processing Systems http://nips.cc/

Wealsousestandard "Big O" notations ( ), !( ), e.g., see [CLRS09, Chapter 3]. In (a),"(4) 0.2" means 4 classesandkµk2/ p p=0.2.

Suppose l is -lipschitzand l-smoothforthefirstargument (i.e.

Empirical risk minimization (ERM) is a crucial framework that offers a general approach to handling a broad range of machine learning tasks.

In this paper, we study this "benign