We show that several machine learning estimators, including square-root least absolute shrinkage and selection and regularized logistic regression, can be represented as solutions to distributionally robust optimization problems. The associated uncertainty regions are based on suitably defined Wasserstein distances. Hence, our representations allow us to view regularization as a result of introducing an artificial adversary that perturbs the empirical distribution to account for out-of-sample effects in loss estimation. In addition, we introduce RWPI (robust Wasserstein profile inference), a novel inference methodology which extends the use of methods inspired by empirical likelihood to the setting of optimal transport costs (of which Wasserstein distances are a particular case). We use RWPI to show how to optimally select the size of uncertainty regions, and as a consequence we are able to choose regularization parameters for these machine learning estimators without the use of cross validation. Numerical experiments are also given to validate our theoretical findings.

In this paper we obtain root-n consistency and functional central limittheorems in weighted L 1-spaces for plug-in estimators of thetwo-step transition density in the classical stationary linear autoregressivemodel of order one, assuming essentially only that the innovation density has bounded variation. We also show that plugging in a properly weighted residual-basedkernel estimator for the unknown innovation densityimproves on plugging in an unweighted residual-based kernel estimator. These weights are chosen to exploit thefact that the innovations have mean zero.If an efficient estimator for the autoregression parameter is used,then the weighted plug-in estimator for the two-step transition density is efficient. Our approach generalizes to invertible linear processes.