论文检索

核岭回归的几何分析及其应用AI 解读

We obtain upper bounds for the estimation error of Kernel Ridge Regression (KRR) for all nonnegative regularization parameters, offering a geometric perspective on various phenomena in KRR. As applications: 1. We address the Multiple Descents problem, unifying the proofs of (In Proceedings of Thirty Third Conference on Learning Theory (2020) 2683–2711 PMLR) and (Ann. Statist. 49 (2021) 1029–1054) for polynomial kernels in nonasymptotic regime and we establish Multiple…

近线性时间内均值向量的稳健次高斯估计AI 解读

We construct an algorithm for estimating the mean of a heavy-tailed random variable when given an adversarial corrupted sample of N independent observations. The only assumption we make on the distribution of the noncorrupted (or informative) data is the existence of a covariance matrix Σ, unknown to the statistician. Our algorithm outputs μˆ, which is robust to the presence of |O| adversarial outliers and satisfies (1)‖μˆ−μ‖2≲ Tr(Σ) N+ ‖Σ‖opK N with probability at least…

基于均值中位数的稳健机器学习：理论与实践AI 解读

Median-of-means (MOM) based procedures have been recently introduced in learning theory (Lugosi and Mendelson (2019); Lecué and Lerasle (2017)). These estimators outperform classical least-squares estimators when data are heavy-tailed and/or are corrupted. None of these procedures can be implemented, which is the major issue of current MOM procedures (Ann. Statist. 47 (2019) 783–794). In this paper, we introduce minmax MOM estimators and show that they achieve the same…

Lipschitz 损失正则化方法的估计界与尖锐预言不等式AI 解读

We obtain estimation error rates and sharp oracle inequalities for regularization procedures of the form \begin{equation*}\hat{f}\in\mathop{\operatorname{argmin}}_{f\in F}\Bigg(\frac{1}{N}\sum_{i=1}^{N}\ell_{f}(X_{i},Y_{i})+\lambda \Vert f\Vert \Bigg)\end{equation*} when $\Vert \cdot \Vert $ is any norm, $F$ is a convex class of functions and $\ell$ is a Lipschitz loss function satisfying a Bernstein condition over $F$. We explore both the bounded and sub-Gaussian stochastic…

Slope 与 Lasso：改进的预言机界及最优性AI 解读

We show that two polynomial time methods, a Lasso estimator with adaptively chosen tuning parameter and a Slope estimator, adaptively achieve the minimax prediction and $\ell_{2}$ estimation rate $(s/n)\log(p/s)$ in high-dimensional linear regression on the class of $s$-sparse vectors in $\mathbb{R}^{p}$. This is done under the Restricted Eigenvalue (RE) condition for the Lasso and under a slightly more constraining assumption on the design for the Slope. The main results…

正则化与小球方法之一：稀疏恢复AI 解读

We obtain bounds on estimation error rates for regularization procedures of the form \begin{equation*}\hat{f}\in\mathop{\operatorname{argmin}}_{f\in F}(\frac{1}{N}\sum_{i=1}^{N}(Y_{i}-f(X_{i}))^{2}+\lambda \Psi(f))\end{equation*} when $\Psi$ is a norm and $F$ is convex. Our approach gives a common framework that may be used in the analysis of learning problems and regularization problems alike. In particular, it sheds some light on the role various notions of sparsity have…