Supervised Feature Selection via Dependence Estimation

The task is to find a functional dependence between x and y, f: x null y, subject to certain optimality conditions. Representative tasks include binary classification, multi-class classification, regression and ranking. We often want to reduce the dimension of the data (the number of features) before the actual learning (Guyon & Elisseeff, 2003); a larger number of features can be associated with higher data collection cost, more difficulty in model interpretation, higher computational cost for the classifier, and decreased generalisationAppearing in Proceedings of the 24 th International Conference on Machine Learning, Corvallis, OR, 2007.