We propose a feature selection method for density estimation withquadratic loss. This method relies on the study of unidimensionalapproximation models and on the definition of confidence regions forthe density thanks to these models. It is quite general and includescases of interest like detection of relevant wavelets coefficientsor selection of support vectors in SVM. In the general case, weprove that every selected feature actually improves the performanceof the estimator. In the case where features are defined bywavelets, we prove that this method is adaptative near minimax (upto a log term) in some Besov spaces. We end the paper bysimulations indicating that it must be possible to extend theadaptation result to other features.
