论文检索

贝叶斯回归树与森林的后验集中AI 解读

Since their inception in the 1980s, regression trees have been one of the more widely used nonparametric prediction methods. Tree-structured methods yield a histogram reconstruction of the regression surface, where the bins correspond to terminal nodes of recursive partitioning. Trees are powerful, yet susceptible to overfitting. Strategies against overfitting have traditionally relied on pruning greedily grown trees. The Bayesian framework offers an alternative remedy…

使用 ReLU 深度神经网络的非参数回归AI 解读

Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log n$-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constraints such as (generalized) additive models. While there is a lot of flexibility…

用收缩方法、实时扩散指数与模型组合预测利率AI 解读

Summary In the context of predicting the term structure of interest rates, we explore the marginal predictive content of real‐time macroeconomic diffusion indexes extracted from a “data rich” real‐time data set, when used in dynamic Nelson–Siegel (NS) models of the variety discussed in Svensson (NBER technical report, 1994; NSS) and Diebold and Li ( Journal of Econometrics , 2006, 130 , 337–364; DNS). Our diffusion indexes are constructed using principal component analysis…

高维广义线性模型的拟合优度检验AI 解读

Summary We propose a family of tests to assess the goodness of fit of a high dimensional generalized linear model. Our framework is flexible and may be used to construct an omnibus test or directed against testing specific non-linearities and interaction effects, or for testing the significance of groups of variables. The methodology is based on extracting left-over signal in the residuals from an initial fit of a generalized linear model. This can be achieved by predicting…

期望低秩随机矩阵的特征向量逐元素分析AI 解读

Recovering low-rank structures via eigenvector perturbation analysis is a common problem in statistical machine learning, such as in factor analysis, community detection, ranking, matrix completion, among others. While a large variety of bounds are available for average errors between empirical and population statistics of eigenvectors, few results are tight for entrywise analyses, which are critical for a number of problems such as community detection. This paper…

基于分区的级数估计量的大样本性质AI 解读

We present large sample results for partitioning-based least squares nonparametric regression, a popular method for approximating conditional expectation functions in statistics, econometrics and machine learning. First, we obtain a general characterization of their leading asymptotic bias. Second, we establish integrated mean squared error approximations for the point estimator and propose feasible tuning parameter selection. Third, we develop pointwise inference methods…

温度化后验及其变分近似的集中性AI 解读

While Bayesian methods are extremely popular in statistics and machine learning, their application to massive data sets is often challenging, when possible at all. The classical MCMC algorithms are prohibitively slow when both the model dimension and the sample size are large. Variational Bayesian methods aim at approximating the posterior by a distribution in a tractable family $\mathcal{F}$. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude…

存在出口时的企业生产率估计：中国制造业证据AI 解读

Summary Motivated by the long‐standing interest of economists in understanding the nexus between firm productivity and export behavior, this paper develops a novel structural framework for control‐function‐based nonparametric identification of the gross production function and latent firm productivity in the presence of endogenous export opportunities that is robust to recent unidentification critiques of proxy estimators. We provide a workable identification strategy,…

双重差分模型中的双重／去偏机器学习AI 解读

Summary This paper provides an orthogonal extension of the semiparametric difference-in-differences estimator proposed in earlier literature. The proposed estimator enjoys the so-called Neyman orthogonality (Chernozhukov et al., 2018), and thus it allows researchers to flexibly use a rich set of machine learning methods in the first-step estimation. It is particularly useful when researchers confront a high-dimensional data set in which the number of potential control…

梯度提升双重机器学习及其对大型审计机构质量效应的应用AI 解读

In this paper, we study the double machine learning (DML) approach of Chernozhukov et al. (2018) for estimating average treatment effect and apply this approach to examine the Big N audit quality effect in the accounting literature. This approach relies on machine learning methods and is suitable when a high dimensional nuisance function with many covariates is present in the model. This approach does not suffer from the “regularization bias” when a learning method with a…

预测、估计与归因AI 解读

The scientific needs and computational limitations of the twentieth century fashioned classical statistical methodology. Both the needs and limitations have changed in the twenty-first, and so has the methodology. Large-scale prediction algorithms—neural nets, deep learning, boosting, support vector machines, random forests—have achieved star status in the popular press. They are recognizable as heirs to the regression tradition, but ones carried out at enormous scale and on…

基于均值中位数的稳健机器学习：理论与实践AI 解读

Median-of-means (MOM) based procedures have been recently introduced in learning theory (Lugosi and Mendelson (2019); Lecué and Lerasle (2017)). These estimators outperform classical least-squares estimators when data are heavy-tailed and/or are corrupted. None of these procedures can be implemented, which is the major issue of current MOM procedures (Ann. Statist. 47 (2019) 783–794). In this paper, we introduce minmax MOM estimators and show that they achieve the same…

利用机器学习估计永久价格冲击AI 解读

In this paper, we show that vector auto-regression (VAR) models, which are commonly used to estimate permanent price impact, are misspecified and can produce conflicting and incorrect inferences when the price impact function is nonlinear. We propose an alternative method to estimate permanent price impact by modifying a reinforcement learning (RL) framework. Our approach assumes the data is stationary and Markov, but is otherwise unrestrictive. We obtain empirical estimates…

多臂老虎机问题：一种有效率的非参数解法AI 解读

Lai and Robbins (Adv. in Appl. Math. 6 (1985) 4–22) and Lai (Ann. Statist. 15 (1987) 1091–1114) provided efficient parametric solutions to the multi-armed bandit problem, showing that arm allocation via upper confidence bounds (UCB) achieves minimum regret. These bounds are constructed from the Kullback–Leibler information of the reward distributions, estimated from specified parametric families. In recent years, there has been renewed interest in the multi-armed bandit…

利用电子健康记录优化个体化治疗策略的匹配学习AI 解读

Current guidelines for treatment decision making largely rely on data from randomized controlled trials (RCTs) studying average treatment effects. They may be inadequate to make individualized treatment decisions in real-world settings. Large-scale electronic health records (EHR) provide opportunities to fulfill the goals of personalized medicine and learn individualized treatment rules (ITRs) depending on patient-specific characteristics from real-world patient data. In…

基于数据深度的非参数缺失值插补AI 解读

We present single imputation method for missing values which borrows the idea of data depth—a measure of centrality defined for an arbitrary point of a space with respect to a probability distribution or data cloud. This consists in iterative maximization of the depth of each observation with missing values, and can be employed with any properly defined statistical depth function. For each single iteration, imputation reverts to optimization of quadratic, linear, or…

从距离相关到多尺度图相关AI 解读

Understanding and developing a correlation measure that can detect general dependencies is not only imperative to statistics and machine learning, but also crucial to general scientific discovery in the big data age. In this paper, we establish a new framework that generalizes distance correlation (Dcorr)—a correlation measure that was recently proposed and shown to be universally consistent for dependence testing against all joint distributions of finite moments—to the…

随机对照试验的最优数据收集AI 解读

Summary In a randomized control trial, the precision of an average treatment effect estimator and the power of the corresponding t-test can be improved either by collecting data on additional individuals, or by collecting additional covariates that predict the outcome variable. To design the experiment, a researcher needs to solve this trade-off subject to her budget constraint. We show that this optimization problem is equivalent to optimally predicting outcomes by the…

This paper proposes a valid bootstrap‐based distributional approximation for M ‐estimators exhibiting a Chernoff (1964)‐type limiting distribution. For estimators of this kind, the standard nonparametric bootstrap is inconsistent. The method proposed herein is based on the nonparametric bootstrap, but restores consistency by altering the shape of the criterion function defining the estimator whose distribution we seek to approximate. This modification leads to a generic and…

实证经济学中正则化估计量的选择：机器学习的风险AI 解读

Abstract Many settings in empirical economics involve estimation of a large number of parameters. In such settings, methods that combine regularized estimation and data-driven choices of regularization parameters are useful. We provide guidance to applied researchers on the choice between regularized estimators and data-driven selection of regularization parameters. We characterize the risk and relative performance of regularized estimators as a function of the…

主特征子空间的分布式估计AI 解读

Principal component analysis (PCA) is fundamental to statistical machine learning. It extracts latent principal factors that contribute to the most variation of the data. When data are stored across multiple machines, however, communication cost can prohibit the computation of PCA in a central location and distributed algorithms for PCA are thus needed. This paper proposes and studies a distributed PCA algorithm: each node machine computes the top $K$ eigenvectors and…

存在中介—结局混杂时路径特定效应的半参数估计AI 解读

Summary Path-specific effects constitute a broad class of mediated effects from an exposure to an outcome via one or more causal pathways along a set of intermediate variables. Most of the literature concerning estimation of mediated effects has focused on parametric models, with stringent assumptions regarding unmeasured confounding. We consider semiparametric inference of a path-specific effect when these assumptions are relaxed. In particular, we develop a suite of…

多重原因的益处AI 解读

Causal inference from observational data is a vital problem, but it comes with strong assumptions. Most methods assume that we observe all confounders, variables that affect both the causal variables and the outcome variables. This assumption is standard but it is also untestable. In this article, we develop the deconfounder, a way to do causal inference with weaker assumptions than the traditional methods require. The deconfounder is designed for problems of multiple causal…

不可知 PAC 机器学习模型的精确下界AI 解读

We provide an exact nonasymptotic lower bound on the minimax expected excess risk (EER) in the agnostic probably-approximately-correct (PAC) machine learning classification model and identify minimax learning algorithms as certain maximally symmetric and minimally randomized “voting” procedures. Based on this result, an exact asymptotic lower bound on the minimax EER is provided. This bound is of the simple form $c_{\infty}/\sqrt{\nu}$ as $\nu\to\infty$, where $c_{\infty}=0.16997\dots$…