Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions

研究问题: 研究问题：在大维度设定（样本量与维度同阶，n ≍ d^γ）下，谱算法的超额风险如何随正则化强度变化，以及良性过拟合在何种条件下出现。摘要称既有大维理论只刻画最优调参点或插值极限，未探索正则化不足区域。
方法与思路: 方法思路：作者先在球面 S^{d-1} 上的内积核，于不同源条件 s ≥ 0 下给出整条正则化路径上超额风险的精确渐近刻画；再将学习曲线分析推广到 R^d 一般区域上满足谱缩放与超压缩条件的一类核的大维 KRR。
材料陈述的贡献: 摘要所述贡献：给出覆盖完整正则化路径的精确渐近刻画，指出学习曲线并非简单 U 形，而分为过正则化、正则化不足与插值三个区域；据此刻画良性过拟合，说明当 s 为正且不超过某临界阈值时，良性过拟合在正则化不足与插值区域一致出现；并说明充分正则化区域的学习曲线可由相应序列模型复现。
阅读关注点: 阅读时应核对：源条件 s 与临界阈值的精确定义、三区域划分的边界条件、良性过拟合的判定标准、序列模型对应关系的成立范围，以及推广到一般区域时谱缩放与超压缩条件的具体形式与验证方式。
阅读边界: 摘要信息的边界：本文为 arXiv 预印本，未经同行评审或期刊接收验证；摘要未说明证明细节、常数条件、有限样本表现与数值实验，也未说明结论对更一般核或数据分布的适用范围。

DeepSeek Flash · 2026-10-13

原文摘要

Existing large-dimensional theory for spectral algorithms resolves either the optimally tuned point or the interpolation limit, but leaves the under-regularized regime unexplored. We study the learning curve and benign overfitting of spectral algorithms in the large-dimensional setting where the sample size and dimension are of comparable order, i.e., $n \asymp d^γ$ for some $γ>0$ . We first consider inner-product kernels on the sphere $\mathbb{S}^{d-1}$ and establish a sharp asymptotic characterization of the excess risk across the full regularization path under various source conditions $s \geq 0$ , where $s$ measures the relative smoothness of the regression function. Our results reveal that the learning curve is not simply U-shaped but instead consists of three distinct regimes: over-regularized, under-regularized, and interpolation regimes. This characterization allows us to fully capture the benign overfitting phenomenon, demonstrating that benign overfitting arises consistently across both the under-regularized and interpolation regimes whenever $s$ is positive but no larger than a critical threshold. We further show that, in the sufficiently regularized regime, the kernel learning curve is recovered by an associated sequence model. Finally, we extend the learning-curve analysis to large-dimensional KRR for a class of kernels on general domains in $\mathbb{R}^d$ whose low-degree eigenspaces satisfy spectral-scaling and hyper-contractivity conditions.Existing large-dimensional theory for spectral algorithms resolves either the optimally tuned point or the interpolation limit, but leaves the under-regularized regime unexplored. We study the learning curve and benign overfitting of spectral algorithms in the large-dimensional setting where the sample size and dimension are of comparable order, i.e., $n \asymp d^γ$ for some $γ>0$. We first consider inner-product kernels on the sphere $\mathbb{S}^{d-1}$ and establish a sharp asymptotic characterization of the excess risk across the full regularization path under various source conditions $s \geq 0$, where $s$ measures the relative smoothness of the regression function. Our results reveal that the learning curve is not simply U-shaped but instead consists of three distinct regimes: over-regularized, under-regularized, and interpolation regimes. This characterization allows us to fully capture the benign overfitting phenomenon, demonstrating that benign overfitting arises consistently across both the under-regularized and interpolation regimes whenever $s$ is positive but no larger than a critical threshold. We further show that, in the sufficiently regularized regime, the kernel learning curve is recovered by an associated sequence model. Finally, we extend the learning-curve analysis to large-dimensional KRR for a class of kernels on general domains in $\mathbb{R}^d$ whose low-degree eigenspaces satisfy spectral-scaling and hyper-contractivity conditions.

摘要来源：arXiv

笔记与分组

分类依据与纠错

Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions

作者的期刊发表依据

代码、数据与补充材料

大维度下谱算法的学习曲线与良性过拟合

原文摘要

同方向的基础文献

同方向的基础文献