Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions

研究问题: 研究高维高斯混合模型下，经验风险关于参数的 Hessian 与信息矩阵的局部几何，包括其经验谱分布极限、离群特征值与特征向量，以及这些谱性质如何随在线 SGD 训练动力学演化。
方法与思路: 作者在 n,d→∞ 且 n/d=φ 的比例渐近框架下，对数据为 k-高斯混合、损失仅通过 x^T Y 依赖数据的设定，推导 Hessian 与信息矩阵的经验谱分布极限及离群特征值、特征向量的精确公式；极限仅通过参数与类均值的 (C+k)×(C+k) Gram 矩阵 G 依赖，并借助已知的有效动力学 ODE 将训练轨迹与测试数据生成的谱理论相连接。
材料陈述的贡献: 摘要称给出了此类矩阵经验谱分布与离群特征值、特征向量极限的精确公式，并指出这些极限仅通过 Gram 矩阵 G 依赖；作者进一步将有效动力学与谱理论相连，并在多类逻辑回归中分析沿有效动力学的有效谱，发现经验 Hessian 与信息矩阵的谱存在显著差异，各自具有静态乃至动态的谱相变。
阅读关注点: 应核对：比例渐近与 i.i.d. 混合模型假设的具体条件；谱极限公式对 G 的依赖是否严格成立；有效动力学 ODE 收敛结论的引用条件与适用范围；多类逻辑回归示例中静态与动态谱相变的定义与推导；C 与 k 不等时结论的稳健性。
阅读边界: 仅依据摘要，无法判断定理的完整识别假设、证明细节、有限样本表现与数值验证；摘要未说明具体收敛速率、相变临界条件的显式形式及与已有谱理论的比较。该文为 arXiv 预印本，未经同行评审，不能据此声称已被期刊接收。

DeepSeek Flash · 2026-10-14

原文摘要

We study the local geometry of empirical risks in high dimensions via the spectral theory of their Hessian and information matrices. We focus on settings where the data, $(Y_\ell)_{\ell =1}^n \in \mathbb{R}^d$ , are i.i.d. draws of a $k$ -Gaussian mixture model, and the loss depends on the projection of the data into a fixed number of vectors, namely $\mathbf{x}^\top Y$ , where $\mathbf{x}\in \mathbb{R}^{d\times C}$ are the parameters, and $C$ need not equal $k$ . This setting captures a broad class of problems such as classification by one and two-layer networks and regression on multi-index models. We provide exact formulas for the limits of the empirical spectral distribution and outlier eigenvalues and eigenvectors of such matrices in the proportional asymptotics limit, where the number of samples and dimension $n,d\to\infty$ and $n/d=φ\in (0,\infty)$ . These limits depend on the parameters $\mathbf{x}$ only through the summary statistic of the $(C+k)\times (C+k)$ Gram matrix of the parameters and class means, $\mathbf{G} = (\mathbf{x},\boldsymbolμ)^\top(\mathbf{x},\boldsymbolμ)$ . It is known that under general conditions, when $\mathbf{x}$ is trained by online stochastic gradient descent, the evolution of these same summary statistics along training converges to the solution of an autonomous system of ODEs, called the effective dynamics. This enables us to connect the training dynamics to the spectral theory of these matrices generated with test data. We demonstrate our general results by analyzing the effective spectrum along the effective dynamics in the case of multi-class logistic regression. In this setting, the empirical Hessian and information matrices have substantially different spectra, each with their own static and even dynamical spectral transitions.We study the local geometry of empirical risks in high dimensions via the spectral theory of their Hessian and information matrices. We focus on settings where the data, $(Y_\ell)_{\ell =1}^n \in \mathbb{R}^d$, are i.i.d. draws of a $k$-Gaussian mixture model, and the loss depends on the projection of the data into a fixed number of vectors, namely $\mathbf{x}^\top Y$, where $\mathbf{x}\in \mathbb{R}^{d\times C}$ are the parameters, and $C$ need not equal $k$. This setting captures a broad class of problems such as classification by one and two-layer networks and regression on multi-index models. We provide exact formulas for the limits of the empirical spectral distribution and outlier eigenvalues and eigenvectors of such matrices in the proportional asymptotics limit, where the number of samples and dimension $n,d\to\infty$ and $n/d=φ\in (0,\infty)$. These limits depend on the parameters $\mathbf{x}$ only through the summary statistic of the $(C+k)\times (C+k)$ Gram matrix of the parameters and class means, $\mathbf{G} = (\mathbf{x},\boldsymbolμ)^\top(\mathbf{x},\boldsymbolμ)$. It is known that under general conditions, when $\mathbf{x}$ is trained by online stochastic gradient descent, the evolution of these same summary statistics along training converges to the solution of an autonomous system of ODEs, called the effective dynamics. This enables us to connect the training dynamics to the spectral theory of these matrices generated with test data. We demonstrate our general results by analyzing the effective spectrum along the effective dynamics in the case of multi-class logistic regression. In this setting, the empirical Hessian and information matrices have substantially different spectra, each with their own static and even dynamical spectral transitions.

摘要来源：arXiv

笔记与分组

分类依据与纠错

Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions

作者的期刊发表依据

代码、数据与补充材料

高维混合模型的局部几何：有效谱理论与动力学相变

原文摘要

同方向的基础文献

同方向的基础文献