Nonparametric sparsity and regularization

It is now common to see practical applications, for example in bioinformatics and computer vision, where the dimensionality of the data is in the order of hundreds, thousands and even tens of thousands. It is known that learning in such a high dimensional regime is feasible only if the quantity to be estimated satisfies some regularity assumptions [24]. In particular, the idea behind, so called, sparsity is that the quantity of interest depends only on a few relevant variables (dimensions). In turn, this latter assumption is often at the basis of the construction of interpretable data models, since the relevant dimensions allow for a compact, hence interpretable, representation. An instance of the above situation is the problem of learning from samples a multivariate function which depends only on a (possibly small) subset of relevant variables. Detecting such variables is the problem of variable selection. Largely motivated by recent advances in compressed sensing [15, 25], the above problem has been extensively studied under the assumption that the function of interest (target function) depends linearly to the relevant variables.