Leveraging Sparsity to Improve No-U-Turn Sampling Efficiency for Hierarchical Bayesian Models

研究问题: 摘要关注的核心问题是：在高维、强相关的分层贝叶斯模型中，NUTS 采样器因后验相关性高而效率下降，如何通过预处理改善其采样效率。
方法与思路: 作者提出 Sparse NUTS（SNUTS），用稀疏精度矩阵 Q 对后验做去相关与去尺度预处理；借助 Template Model Builder 从边际后验的拉普拉斯近似众数处计算 Q，再通过 Stan 对预处理后的后验采样。
材料陈述的贡献: 摘要称在十七个案例研究中，以 Q 预处理比 Stan 的行业标准对角或稠密预处理收敛快一到两个数量级，也优于用 Pathfinder 变分推断所得协方差逆的预处理；在高维、稀疏、强相关情形最有优势，可扩展到超过 10^4 个参数，并提供 R 包 SparseNUTS 开源实现。
阅读关注点: 应核对十七个案例的具体设定与基准比较方式、收敛速度的度量口径、Q 的构造与拉普拉斯近似条件、可扩展性证据，以及作者自述在漏斗形、厚尾或多峰等高变曲率模型上无改善的边界。
阅读边界: 摘要未说明各案例的数据来源、参数规模分布、收敛判据与误差控制细节，也未给出理论保证；所报告的一到两个数量级加速仅为作者摘要陈述，需读全文核实。

DeepSeek Flash · 2026-10-14

原文摘要

Analysts routinely use Bayesian hierarchical models to understand natural processes. The no-U-turn sampler (NUTS) is the most widely used algorithm to sample high-dimensional, continuously differentiable models. But NUTS is slowed by high correlations, especially in high dimensions, limiting the complexity of applied analyses. Here we introduce Sparse NUTS (SNUTS), which preconditions (decorrelates and descales) posteriors using a sparse precision matrix ( $Q$ ). We use Template Model Builder (TMB) to efficiently compute $Q$ from the mode of the Laplace approximation to the marginal posterior, then pass the preconditioned posterior to NUTS through the Bayesian software Stan for sampling. We apply SNUTS to seventeen diverse case studies to demonstrate that preconditioning with $Q$ converges one to two orders of magnitude faster than Stan's industry standard diagonal or dense preconditioners. SNUTS also outperforms preconditioning with the inverse of the covariance estimated with Pathfinder variational inference. SNUTS does not improve sampling efficiency for models with the highly varying curvature found in funnels, wide tails, or multiple modes. SNUTS is most advantageous, and can be scaled beyond $10^4$ parameters, in the presence of high dimensionality, sparseness, and high correlations, all of which are widespread in applied statistics. An open-source implementation of SNUTS is provided in the R package SparseNUTS.Analysts routinely use Bayesian hierarchical models to understand natural processes. The no-U-turn sampler (NUTS) is the most widely used algorithm to sample high-dimensional, continuously differentiable models. But NUTS is slowed by high correlations, especially in high dimensions, limiting the complexity of applied analyses. Here we introduce Sparse NUTS (SNUTS), which preconditions (decorrelates and descales) posteriors using a sparse precision matrix ($Q$). We use Template Model Builder (TMB) to efficiently compute $Q$ from the mode of the Laplace approximation to the marginal posterior, then pass the preconditioned posterior to NUTS through the Bayesian software Stan for sampling. We apply SNUTS to seventeen diverse case studies to demonstrate that preconditioning with $Q$ converges one to two orders of magnitude faster than Stan's industry standard diagonal or dense preconditioners. SNUTS also outperforms preconditioning with the inverse of the covariance estimated with Pathfinder variational inference. SNUTS does not improve sampling efficiency for models with the highly varying curvature found in funnels, wide tails, or multiple modes. SNUTS is most advantageous, and can be scaled beyond $10^4$ parameters, in the presence of high dimensionality, sparseness, and high correlations, all of which are widespread in applied statistics. An open-source implementation of SNUTS is provided in the R package SparseNUTS.

摘要来源：arXiv

笔记与分组

分类依据与纠错

Leveraging Sparsity to Improve No-U-Turn Sampling Efficiency for Hierarchical Bayesian Models

作者的期刊发表依据

代码、数据与补充材料

利用稀疏性提升分层贝叶斯模型的No-U-Turn采样效率

原文摘要

同方向的基础文献

同方向的基础文献