Analyzing bagging

Bagging is one of the most effective computationally intensive procedures to improve on unstable estimators or classifiers, useful especially for high dimensional data set problems. Here we formalize the notion of instability and derive theoretical results to analyze the variance reduction effect of bagging (or variants thereof) in mainly hard decision problems, which include estimation after testing in regression and decision trees for regression functions and classifiers. Hard decisions create instability, and bagging is shown to smooth such hard decisions, yielding smaller variance and mean squared error. With theoretical explanations, we motivate subagging based on subsampling as an alternative aggregation scheme. It is computationally cheaper but still shows approximately the same accuracy as bagging. Moreover, our theory reveals improvements in first order and in line with simulation studies. In particular, we obtain an asymptotic limiting distribution at the cube-root rate for the split point when fitting piecewise constant functions. Denoting sample size by n, it follows that in a cylindric neighborhood of diameter $n^{-1/3}$ of the theoretically optimal split point, the variance and mean squared error reduction of subagging can be characterized analytically. Because of the slow rate, our reasoning also provides an explanation on the global scale for the whole covariate space in a decision tree with finitely many splits.Bagging is one of the most effective computationally intensive procedures to improve on unstable estimators or classifiers, useful especially for high dimensional data set problems. Here we formalize the notion of instability and derive theoretical results to analyze the variance reduction effect of bagging (or variants thereof) in mainly hard decision problems, which include estimation after testing in regression and decision trees for regression functions and classifiers. Hard decisions create instability, and bagging is shown to smooth such hard decisions, yielding smaller variance and mean squared error. With theoretical explanations, we motivate subagging based on subsampling as an alternative aggregation scheme. It is computationally cheaper but still shows approximately the same accuracy as bagging. Moreover, our theory reveals improvements in first order and in line with simulation studies. In particular, we obtain an asymptotic limiting distribution at the cube-root rate for the split point when fitting piecewise constant functions. Denoting sample size by n, it follows that in a cylindric neighborhood of diameter $n^{-1/3}$ of the theoretically optimal split point, the variance and mean squared error reduction of subagging can be characterized analytically. Because of the slow rate, our reasoning also provides an explanation on the global scale for the whole covariate space in a decision tree with finitely many splits.

摘要来源：OpenAlex 索引摘要（稿件版本未注明）

笔记与分组

同方向的基础文献

相关领域论文

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

A unified spectral analysis of high-dimensional linear regression: debiasing and thresholding

Ying-Ao Wang · Yunyi Zhang · Ye Zhang首次提交 2025-02-24 · 修订 2026-10-12 · v3arXiv预印本

高维线性回归的统一谱分析：去偏与阈值化AI 解读

We use spectral analysis to establish a unified theory of debiasing, thresholding, and simultaneous inference for high-dimensional linear regression with ill-conditioned designs. The methods include Ridge regression, spectral cut-off, and iterative and dynamical regression procedures. A common debiasing procedure uses two applications of the original linear algorithm with fixed parameters. Spectral analysis shows that debiasing squares the residual, while implementations…

正则化与变量选择 GMM 与矩条件 Bootstrap 与重抽样

被引：暂无数据

AI 解读阅读原文 ↗BibTeX ↓

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

Cluster-Robust Prediction-Powered Inference

David Broska · Michael Howes首次提交 2026-10-07 · 修订 2026-10-07 · v1arXiv预印本

聚类稳健的预测辅助推断AI 解读

Data collection is often costly or logistically demanding, limiting both the questions researchers can pursue and how precisely they can answer them. Prediction-powered inference (PPI) can reduce the amount of data needed for precise parameter estimation by combining labeled data with machine learning predictions. However, ignoring dependence within clusters can produce confidence intervals that cover the true parameter less often than their nominal rate. We introduce…

稳健标准误机器学习与 DML Bootstrap 与重抽样

被引：暂无数据

AI 解读阅读原文 ↗BibTeX ↓

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

Fridge: focused fine-tuning of ridge regression for personalize predictions

Kristoffer Herland Hellton · Nils Lid Hjort首次提交 2026-10-06 · 修订 2026-10-06 · v1arXiv预印本

Fridge：面向个体预测的岭回归聚焦调参AI 解读

Statistical prediction methods typically require some form of fine-tuning of tuning parameter(s), with $K$-fold cross-validation as the canonical procedure. For ridge regression there exist numerous procedures, but common for all, including cross-validation, is that one single parameter is chosen for all future predictions. We propose instead to calculate a unique tuning parameter for each individual for which we wish to predict an outcome. This generates an individualized…

正则化与变量选择 Bootstrap 与重抽样

被引：暂无数据

AI 解读阅读原文 ↗BibTeX ↓

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

Inference for Standard Trial Estimands after Data-Driven Subgroup Discovery

Larry F. Leon · Keaven M. Anderson首次提交 2026-09-29 · 修订 2026-09-29 · v1arXiv预印本

数据驱动亚组发现后标准试验估计量的推断AI 解读

Data-driven subgroup identification is increasingly used in clinical trials, yet the effect reported for a discovered subgroup is typically the standard analysis: a Cox or generalized-linear-model treatment coefficient fitted within that subgroup. Selection by an effect-driven rule --- rewarding large estimated effects subject to screening and size criteria --- inflates the reported coefficient: a winner's curse invalidating naive intervals whether or not the identifier's…

微观 · 因果推断机器学习与 DML Bootstrap 与重抽样

被引：暂无数据

AI 解读阅读原文 ↗BibTeX ↓

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

Debiased Inference for Bounding Wage Inequality with Many Controls

Yaroslav Korobka · Vira Semenova首次提交 2026-09-29 · 修订 2026-09-29 · v1arXiv预印本

多控制变量下工资不平等界的去偏推断AI 解读

We study estimation and inference for a partially identified parameter whose identified set depends on a first-stage nuisance parameter that must itself be estimated. Combining the criterion-function approach with the theory of Neyman-orthogonal moments that underlies double/debiased machine learning, we propose a two-step procedure: the point-identified nuisance is estimated by flexible machine-learning methods, and the set-identified target is recovered as a level set of a…

微观 · 因果推断机器学习与 DML 部分识别与矩不等式

关联记录被引 0 次 ↗OpenAlex · 核验 2026-10-01

AI 解读阅读原文 ↗BibTeX ↓

共同方法：高维与机器学习、检验与推断、一般估计、Bootstrap 与重抽样

Debiased and Simultaneous Inference for Heterogeneous Factorial Effect Modifiers via Residualized Walsh-Hadamard Scores

Tomoshige Nakamura · Ryo Emoto首次提交 2026-09-29 · 修订 2026-09-29 · v1arXiv预印本

基于残差化 Walsh–Hadamard 得分的异质析因效应修饰因子去偏与同时推断AI 解读

In factorial randomized experiments with $K$ binary treatment components, we study which baseline covariates modify component main effects and interactions. We define effect-modifier cells $(j,S)$, $j\in[p]$ and $\emptyset\neq S\subseteq[K]$, through the coefficients $θ^\ast_{jS}$ of population linear projections of conditional Walsh--Hadamard contrasts $τ_{S}$ onto standardized covariates. Using baseline-robust direct scores, we develop frequentist inference for all $p(2^K-1)$…

微观 · 因果推断多重检验与同时推断正则化与变量选择

关联记录被引 0 次 ↗OpenAlex · 核验 2026-10-08

AI 解读阅读原文 ↗BibTeX ↓