论文检索

经验风险最小化中的梯度下降推断AI 解读

Gradient descent is one of the most widely used iterative algorithms in modern statistical learning. However, its precise algorithmic dynamics in high-dimensional settings remain only partially understood, which has limited its broader potential for statistical inference applications. This paper provides a precise, nonasymptotic joint distributional characterization of gradient descent iterates and their debiased statistics in a broad class of empirical risk minimization…

一般一阶方法的逐坐标动态与普适性AI 解读

General first-order methods (GFOMs), including many variants of gradient descent and approximate message passing algorithms, constitute a broad class of iterative algorithms widely applied in modern statistical learning problems. Some GFOMs also serve as constructive proof devices, iteratively characterizing the empirical distributions of statistical estimators in the asymptotic regime of large system limits for any fixed number of iterations. This paper develops a…

凸性约束模型中局部参数的统计推断AI 解读

In this article, we develop automated inference methods for “local” parameters in a collection of convexity constrained models based on the natural constrained tuning-free estimators. A canonical example is given by the univariate convex regression model, in which automated inference is drawn for the function value, the function derivative at a fixed interior point, and the anti-mode of the convex regression function, based on the widely used tuning-free, piecewise linear…

高维正则化回归估计量的普适性AI 解读

The Convex Gaussian Min–Max Theorem (CGMT) has emerged as a prominent theoretical tool for analyzing the precise stochastic behavior of various statistical estimators in the so-called high-dimensional proportional regime, where the sample size and the signal dimension are of the same order. However, a well-recognized limitation of the existing CGMT machinery rests in its stringent requirement on the exact Gaussianity of the design matrix, therefore rendering the obtained…

凸约束下含噪线性逆问题的高维精确风险渐近AI 解读

In the standard Gaussian linear measurement model Y=Xμ0+ξ∈Rm with a fixed noise level σ>0, we consider the problem of estimating the unknown signal μ0 under a convex constraint μ0∈K, where K is a closed convex set in Rn. We show that the risk of the natural convex constrained least squares estimator (LSE) μˆ(σ) can be characterized exactly in high-dimensional limits, by that of the convex constrained LSE μˆKseq in the corresponding Gaussian sequence model at a different…

S 形函数的非参数免调参估计AI 解读

Abstract We consider the nonparametric estimation of an S-shaped regression function. The least squares estimator provides a very natural, tuning-free approach, but results in a non-convex optimization problem, since the inflection point is unknown. We show that the estimator may nevertheless be regarded as a projection onto a finite union of convex cones, which allows us to propose a mixed primal-dual bases algorithm for its efficient, sequential computation. After…

凸约束高斯序列模型中似然比检验的高维渐近理论AI 解读

In the Gaussian sequence model Y=μ+ξ, we study the likelihood ratio test (LRT) for testing H0:μ=μ0 versus H1:μ∈K, where μ0∈K, and K is a closed convex set in Rn. In particular, we show that under the null hypothesis, normal approximation holds for the log-likelihood ratio statistic for a general pair (μ0,K), in the high-dimensional regime where the estimation error of the associated least squares estimator diverges in an appropriate sense. The normal approximation further…

具有集合结构的全局经验风险最小化估计在一般维度下达到最优速率AI 解读

Entropy integrals are widely used as a powerful empirical process tool to obtain upper bounds for the rates of convergence of global empirical risk minimizers (ERMs), in standard settings such as density estimation and regression. The upper bound for the convergence rates thus obtained typically matches the minimax lower bound when the entropy integral converges, but admits a strict gap compared to the lower bound when it diverges. Birgé and Massart (Probab. Theory Related…

多维保序回归及其他单调模型的置信区间AI 解读

We consider the problem of constructing pointwise confidence intervals in the multiple isotonic regression model. Recently, Han and Zhang (2020) obtained a pointwise limit distribution theory for the so-called block max–min and min–max estimators (Fokianos, Leucht and Neumann (2020); Deng and Zhang (2020)) in this model, but inference remains a difficult problem due to the nuisance parameter in the limit distribution that involves multiple unknown partial derivatives of the…

复杂抽样设计：一致极限定理及其应用AI 解读

In this paper, we develop a general approach to proving global and local uniform limit theorems for the Horvitz–Thompson empirical process arising from complex sampling designs. Global theorems such as Glivenko–Cantelli and Donsker theorems, and local theorems such as local asymptotic modulus and related ratio-type limit theorems are proved for both the Horvitz–Thompson empirical process, and its calibrated version. Limit theorems of other variants and their conditional…

多元保序回归块估计量的极限分布理论AI 解读

We study limit distributions for the tuning-free max–min block estimator originally proposed in (Fokianos, Leucht and Neumann (2017)) in the problem of multiple isotonic regression, under both fixed lattice design and random design settings. We show that, if the regression function $f_{0}$ admits vanishing derivatives up to order $\alpha_{k}$ along the $k$th dimension ($k=1,\ldots,d$) at a fixed point $x_{0}\in(0,1)^{d}$, and the errors have variance $\sigma^{2}$, then the…

一般维度下的保序回归AI 解读

We study the least squares regression function estimator over the class of real-valued functions on $[0,1]^{d}$ that are increasing in each coordinate. For uniformly bounded signals and with a fixed, cubic lattice design, we establish that the estimator achieves the minimax rate of order $n^{-\min\{2/(d+2),1/d\}}$ in the empirical $L_{2}$ loss, up to polylogarithmic factors. Further, we prove a sharp oracle inequality, which reveals in particular that when the true…

重尾误差下最小二乘回归估计量的收敛速度AI 解读

We study the performance of the least squares estimator (LSE) in a general nonparametric regression model, when the errors are independent of the covariates but may only have a $p$th moment ($p\geq1$). In such a heavy-tailed regression setting, we show that if the model satisfies a standard “entropy condition” with exponent $\alpha\in(0,2)$, then the $L_{2}$ loss of the LSE converges at a rate \[\mathcal{O}_{\mathbf{P}}\bigl(n^{-\frac{1}{2+\alpha}}\vee n^{-\frac{1}{2}+\frac{1}{2p}}\bigr).\]…

基于Rényi散度的s凹密度逼近与估计AI 解读

In this paper, we study the approximation and estimation of $s$-concave densities via Rényi divergence. We first show that the approximation of a probability measure $Q$ by an $s$-concave density exists and is unique via the procedure of minimizing a divergence functional proposed by [Ann. Statist. 38 (2010) 2998–3027] if and only if $Q$ admits full-dimensional support and a first moment. We also show continuity of the divergence functional in $Q$: if $Q_{n}\to Q$ in the…