Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon
路径正则化：多层神经网络与双下降的非渐近泛化理论AI 解读 ↓
← 返回论文列表与筛选
阅读说明
本页整理公开书目信息，不生成学术质量评分。AI 解读根据题名及所注明的摘要或原文片段生成，未完成全文核验，可能有误；具体假设、证明与结论请核对原文。摘要保留来源文本，未经模型改写。
DOI 首次登记不等于接收或发表日期。部分更正、评论等条目没有独立摘要；缺少可靠材料时保留待补状态。
本页为预印本版本；arXiv 收录、作者填写的期刊信息以及作者既有发表经历，都不单独证明本篇已接收。作者发表记录按 OpenAlex 身份与完整姓名、本站覆盖期刊 ISSN 及正式卷期核对，只展示已核实例，档案仍可能误合并。
引用量采用 OpenAlex 单条关联记录，不合计多个记录；关联记录可能包含正式发表版本，不等于此版本单独引用。缺失不视为零，引用指标不代表论文质量。
关注作者按书目中的完整姓名匹配，同名和姓名变体需人工辨别。相关论文先按共享细分方向和具体方法排列，再参考通用方法与发表日期。
分类依据与纠错
AI 根据所提供的论文材料归类：摘要研究一般神经网络泛化与逼近理论，未明确提供所列经济计量应用方向的证据。
反馈分类问题
正在核对关联版本…
代码、数据与补充材料
正在核对对应版本的资源…
关注这篇论文的作者
路径正则化：多层神经网络与双下降的非渐近泛化理论
- 研究问题
- 路径正则化如何影响多层神经网络的有限样本泛化，能否在无需损失有界或网络规模趋于无穷的条件下刻画双下降现象？
- 方法与思路
- 作者为满足所述激活条件及广泛Lipschitz损失的网络建立含逼近误差的显式泛化上界，并研究广义Barron空间逼近与ReLU回归。
- 材料陈述的贡献
- 摘要称给出广义Barron空间逼近率及ReLU回归的近极小极大结果，误差上界呈双下降；匹配下界被列为后续工作。
- 阅读关注点
- 应核对激活与损失条件、函数类和路径范数约束、近最优性的比较对象，以及误差上界的双下降形状与实际泛化风险的关系。
- 阅读边界
- 依据arXiv v3预印本摘要，同行评审或期刊接收未核验；匹配下界尚列为后续工作，不能把上界形状当作已证实的双下降机制。
原文摘要
Path regularization has shown to be a very effective regularization to train neural networks, leading to a better generalization property than common regularizations i.e. weight decay, etc. We propose a first near-complete (as will be made explicit in the main text) nonasymptotic generalization theory for multilayer neural networks with path regularizations for general learning problems. In particular, it does not require the boundedness of the loss function, as is commonly assumed in the literature. Our theory goes beyond the bias-variance tradeoff and aligns with phenomena typically encountered in deep learning. It is therefore sharply different from other existing nonasymptotic generalization error bounds. More explicitly, we propose an explicit generalization error upper bound for multilayer neural networks with $σ(0)=0$ and sufficiently broad Lipschitz loss functions, without requiring the width, depth, or other hyperparameters of the neural network to approach infinity, a specific neural network architecture (e.g., sparsity), or boundedness of the loss function, while also taking approximation error into consideration. In particular, we solve an open problem proposed by Weinan E et. al. in 2020 regarding the approximation rates in generalized Barron spaces. Furthermore, we show the near-minimax optimality of our theory for regression problems with ReLU activations. Notably, our upper bound exhibits the famous double descent phenomenon for such networks, which is the most distinguished characteristic compared with other existing results. Our subsequent work will prove the matching lower bounds in the minimax sense, meaning that it is highly possible that our theory reveals the true underlying mechanism of the double descent phenomenon. We can also explain scaling law from this theory.