论文检索

线性模型迭代算法的不确定性量化及其早停应用AI 解读

This paper investigates the iterates bˆ1,…,bˆT obtained from iterative algorithms in high-dimensional linear regression problems, in the regime where the feature dimension p is comparable with the sample size n, that is, p≍n. The analysis and proposed estimators are applicable to Gradient Descent (GD), proximal GD and their accelerated variants such as Fast Iterative Soft-Thresholding (FISTA). The paper proposes novel estimators for the generalization error of the iterate…

正则化M估计量装袋集成的精确渐近分析AI 解读

We characterize the squared prediction risk of ensemble estimators obtained through subagging (subsample bootstrap aggregating) regularized M-estimators and construct a consistent estimator for the risk. Specifically, we consider a heterogeneous collection of M≥1 regularized M-estimators, each trained with (possibly different) subsample sizes, convex differentiable losses, and convex regularizers. We operate under the proportional asymptotics regime, where the sample size n,…

p/n 有界时单指标模型中正则化 M 估计的可观测调整AI 解读

We consider observations (X,y) from single index models with unknown link function, Gaussian covariates and a regularized M-estimator βˆ constructed from convex loss function and regularizer. In the regime where sample size n and dimension p are both increasing such that p/n has a finite limit, the behavior of the empirical distribution of βˆ and the predicted values Xβˆ has been previously characterized in a number of models: The empirical distributions are known to…

凸正则化估计量的去偏与线性模型区间估计AI 解读

New upper bounds are developed for the L2 distance between ξ/Var[ξ]1/2 and linear and quadratic functions of z∼N(0,In) for random variables of the form ξ=z⊤f(z)−divf(z). The linear approximation yields a central limit theorem when the squared norm of f(z) dominates the squared Frobenius norm of ∇f(z) in expectation. Applications of this normal approximation are given for the asymptotic normality of debiased estimators in linear regression with correlated design and convex…

二阶 Stein 公式：SURE 的风险估计及高维推断应用AI 解读

Stein’s formula states that a random variable of the form z⊤f(z)−divf(z) is mean-zero for all functions f with integrable gradient. Here, divf is the divergence of the function f and z is a standard normal vector. This paper aims to propose a second-order Stein formula to characterize the variance of such random variables for all functions f(z) with square integrable gradient, and to demonstrate the usefulness of this second-order Stein formula in various applications. In…

Slope 与 Lasso：改进的预言机界及最优性AI 解读

We show that two polynomial time methods, a Lasso estimator with adaptively chosen tuning parameter and a Slope estimator, adaptively achieve the minimax prediction and $\ell_{2}$ estimation rate $(s/n)\log(p/s)$ in high-dimensional linear regression on the class of $s$-sparse vectors in $\mathbb{R}^{p}$. This is done under the Restricted Eigenvalue (RE) condition for the Lasso and under a slightly more constraining assumption on the design for the Slope. The main results…

形状约束回归中最小二乘估计的精确预言不等式AI 解读

The performance of Least Squares (LS) estimators is studied in shape-constrained regression models under Gaussian and sub-Gaussian noise. General bounds on the performance of LS estimators over closed convex sets are provided. These results have the form of sharp oracle inequalities that account for the model misspecification error. In the presence of misspecification, these bounds imply that the LS estimator estimates the projection of the true parameter at the same rate as…

仿射估计量聚合的最优界AI 解读

We study the problem of aggregation of estimators when the estimators are not independent of the data used for aggregation and no sample splitting is allowed. If the estimators are deterministic vectors, it is well known that the minimax rate of aggregation is of order $\log(M)$, where $M$ is the number of estimators to aggregate. It is proved that for affine estimators, the minimax rate of aggregation is unchanged: it is possible to handle the linear dependence between the…