The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression
对抗训练中的过度参数化之弊：随机特征回归的稳健泛化分析AI 解读 ↓
← 返回论文列表与筛选
阅读说明
本页整理公开书目信息，不生成学术质量评分。AI 解读根据题名及所注明的摘要或原文片段生成，未完成全文核验，可能有误；具体假设、证明与结论请核对原文。摘要保留来源文本，未经模型改写。
DOI 首次登记不等于接收或发表日期。部分更正、评论等条目没有独立摘要；缺少可靠材料时保留待补状态。
引用量采用 OpenAlex 单条关联记录，不合计多个记录；关联记录可能包含正式发表版本，不等于此版本单独引用。缺失不视为零，引用指标不代表论文质量。
关注作者按书目中的完整姓名匹配，同名和姓名变体需人工辨别。相关论文先按共享细分方向和具体方法排列，再参考通用方法与发表日期。
分类依据与纠错
AI 根据所提供的论文材料归类：摘要研究机器学习模型的稳健泛化，没有收益预测、宏观预测或其他所列领域的直接证据。
反馈分类问题
正在核对关联版本…
代码、数据与补充材料
正在核对对应版本的资源…
关注这篇论文的作者
对抗训练中的过度参数化之弊：随机特征回归的稳健泛化分析
- 研究问题
- 在对抗训练的随机特征回归中，参数数量超过样本数量如何影响受扰动输入上的泛化误差，过度参数化是否仍然有益？
- 方法与思路
- 作者研究第一层权重随机的双层网络，在样本量、输入维度及参数量按比例增长的渐近框架下，刻画稳健泛化误差。
- 材料陈述的贡献
- 摘要称，作者推导稳健泛化误差的渐近精确公式，揭示过度参数化对稳健性的非平凡影响，并指出高度过度参数化可能有害。
- 阅读关注点
- 应核对输入与响应的生成条件、对抗扰动集合、训练目标及比例增长设定，并检查有害效应出现于哪些参数范围。
- 阅读边界
- 仅据版本未注明的索引摘要；摘要未说明扰动约束及具体模型条件，结果不能直接推广到任意深度网络或有限样本训练场景。
原文摘要
Successful deep learning models often involve training neural network architectures that contain more parameters than the number of training samples. Such overparametrized models have recently been extensively studied, and the virtues of overparametrization have been established from both the statistical perspective, via the double-descent phenomenon, and the computational perspective via the structural properties of the optimization landscape. Despite this success, it is also well known that these models are highly vulnerable to small adversarial perturbations in their inputs. Even when adversarially trained, their performance on perturbed inputs (robust generalization) is considerably worse than their best attainable performance on benign inputs (standard generalization). It is thus imperative to understand how overparametrization fundamentally affects robustness. In this paper, we will provide a precise characterization of the role of overparametrization on robustness by focusing on random features regression models (two-layer neural networks with random first layer weights). We consider a regime where the sample size, the input dimension and the number of parameters grow proportionally, and derive an asymptotically exact formula for the robust generalization error when the model is adversarially trained. Our developed theory reveals the nontrivial effect of overparametrization on robustness and indicates that high overparametrization can hurt robust generalization.