Extensions of stability selection using subsamples of observations and covariates

Variable selection techniques aim at identifying such relevant covariates (for a review see Guyon, 2006). Usually, variable selection aims at one of two goals: to identify informative covariates in order to get scientific insight into the data and the process that generated the outcome; or to use the covariates identified as relevant in order to predict the outcome. In this work we primarily focus on the identification of informative covariates but also consider prediction results using real data. We consider variable selection (also called feature selection in computer science-related communities) as a part of the broader field of dimensionality reduction. Many variable selection methods share the common drawback of being unstable with respect to small changes of the data: if one estimates the set of relevant covariates on different sets of observations coming from the same source, the result can vary significantly.