Rodeo: Sparse Nonparametric Regression in High Dimensions

We present a method for nonparametric regression that performs bandwidth selectionand variable selection simultaneously. The approach is based on the technique of incrementally decreasing the bandwidth in directions wherethe gradient of the estimator with respect to bandwidth is large. When the unknown function satisfies a sparsity condition, our approach avoids the curse of dimensionality, achieving the optimal minimax rateof convergence, up to logarithmic factors, as if the relevant variables wereknown in advance. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests, and is easy to implement. A modified version that replaces hard with soft thresholding effectively solves a sequence of lasso problems.