Kernel Feature Selection via Conditional Covariance Minimization

Feature selection is an important problem in statistical machine learning, and is a common method for dimensionality reduction that encourages model interpretability. With large data sets becoming ever more prevalent, feature selection has seen widespread usage across a variety of real-world tasks in recent years, including text classification, gene selection from microarray data, and face recognition [3, 13, 17]. In this work, we consider the supervised variant of feature selection, which entails finding a subset of the input features that explains the output well. This practice can reduce the computational expense of downstream learning by removing features that are redundant or noisy, while simultaneously providing insight into the data through the features that remain. Feature selection algorithms can generally be divided into three main categories: filter methods, wrapper methods, and embedded methods [13].