Class-size Independent Generalization Analsysis of Some Discriminative Multi-Category Classification

We consider the problem of deriving class-size independent generalization boundsfor some regularized discriminative multi-category classification methods.In particular, we obtain an expected generalization bound for a standard formulation of multi-category support vector machines. Basedon the theoretical result, we argue that the formulation over-penalizes misclassification error, which in theory may lead to poor generalization performance. A remedy, based on a generalization of multi-category logistic regression (conditional maximum entropy), is then proposed, and its theoretical properties are examined.