Bounds on the Bayes Error Given Moments

A standard approach in pattern recognition is to estimate the first two moments of each class-conditional distribution from training samples, and then assume the unknown distributions are Gaussians. Depending on the exact assumptions, this approach is called linear or quadratic discriminant analysis (QDA) [1], [2]. Gaussians are known to maximize entropy given the first two moments [3] and to have other nice mathematical properties, but how robust are they with respect to maximizing the Bayes error? To answer that, in this paper we investigate the more general question: "What is the maximum possible Bayes error given moment constraints on the class-conditional distributions?" We present both a lower bound and an upper bound for the maximum possible Bayes error.