论文检索

条件平均处理效应及其他因果函数的去偏机器学习AI 解读

Summary This paper provides estimation and inference methods for the best linear predictor (approximation) of a structural function, such as conditional average structural and treatment effects, and structural derivatives, based on modern machine learning tools. We represent this structural function as a conditional expectation of an unbiased signal that depends on a nuisance parameter, which we estimate by modern machine learning techniques. We first adjust the signal to…

通过分布鲁棒优化学习表现稳定的模型AI 解读

A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts or unmodeled temporal effects. We develop and analyze a distributionally robust stochastic optimization (DRO) framework that learns a model providing good performance against perturbations to the data-generating distribution. We give a convex formulation for the problem, providing…

异质性处理效应的准 Oracle 估计AI 解读

Summary Flexible estimation of heterogeneous treatment effects lies at the heart of many statistical applications, such as personalized medicine and optimal resource allocation. In this article we develop a general class of two-step algorithms for heterogeneous treatment effect estimation in observational studies. First, we estimate marginal effects and treatment propensities to form an objective function that isolates the causal component of the signal. Then, we optimize…

用机器学习刻画参数区域AI 解读

How many random points from an identified set, a confidence set, or a highest posterior density set suffice to describe them? This paper argues that taking random draws from a parameter region in order to approximate its shape is a supervised learning problem (analogous to sampling pixels of an image to recognize it). Misclassification error – a common criterion in machine learning – provides an off-the-shelf tool to assess the quality of a given approximation. We say a…

带正则化的最大线性回归模型AI 解读

Motivated by the newly developed max-linear competing copula factor models and max-stable nonlinear time series models, we propose a new class of max-linear regression models to take advantages of easy interpretable features embedded in linear regression models. It can be seen that linear relation is a special case of max-linear relation. We develop an EM algorithm based maximum likelihood estimation procedure. The consistency and asymptotics of the estimators for parameters…

多项 Logistic 与计数回归的对数线性贝叶斯加性回归树AI 解读

We introduce Bayesian additive regression trees (BART) for log-linear models including multinomial logistic regression and count regression with zero-inflation and overdispersion. BART has been applied to nonparametric mean regression and binary classification problems in a range of settings. However, existing applications of BART have been mostly limited to models for Gaussian “data,” either observed or latent. This is primarily because efficient MCMC algorithms are…

人口特征与社交距离指标在美国COVID-19病例深度预测中的作用AI 解读

With the severity of the COVID-19 outbreak, we characterize the nature of the growth trajectories of counties in the United States using a novel combination of spectral clustering and the correlation matrix. As the United States and the rest of the world are still suffering from the effects of the virus, the importance of assigning growth membership to counties and understanding the determinants of the growth is increasingly evident. For the two communities (faster versus…

保护隐私的参数推断：稳健统计的作用AI 解读

Differential privacy is a cryptographically motivated approach to privacy that has become a very active field of research over the last decade in theoretical computer science and machine learning. In this paradigm, one assumes there is a trusted curator who holds the data of individuals in a database and the goal of privacy is to simultaneously protect individual data while allowing the release of global characteristics of the database. In this setting, we introduce a…

异质性因果效应的机器学习估计：经验蒙特卡洛证据AI 解读

Summary We investigate the finite-sample performance of causal machine learning estimators for heterogeneous causal effects at different aggregation levels. We employ an empirical Monte Carlo study that relies on arguably realistic data generation processes (DGPs) based on actual data in an observational setting. We consider 24 DGPs, eleven causal machine learning estimators, and three aggregation levels of the estimated effects. Four of the considered estimators perform…

无限维、零直接样本情形的多分辨率近似理论：过渡推断与个体化预测AI 解读

Transitional inference is an empiricism concept, rooted and practiced in clinical medicine since ancient Greece. Knowledge and experiences gained from treating one entity (e.g., a disease or a group of patients) are applied to treat a related but distinctively different one (e.g., a similar disease or a new patient). This notion of “transition to the similar” renders individualized treatments an operational meaning, yet its theoretical foundation defies the familiar…

数据丰富环境下的通胀预测：机器学习方法的益处AI 解读

Inflation forecasting is an important but difficult task. Here, we explore advances in machine learning (ML) methods and the availability of new datasets to forecast U.S. inflation. Despite the skepticism in the previous literature, we show that ML models with a large number of covariates are systematically more accurate than the benchmarks. The ML method that deserves more attention is the random forest model, which dominates all other models. Its good performance is due…

黑箱模型的因果解释AI 解读

The fields of machine learning and causal inference have developed many concepts, tools, and theory that are potentially useful for each other. Through exploring the possibility of extracting causal interpretations from black-box machine-trained models, we briefly review the languages and concepts in causal inference that may be interesting to machine learning researchers. We start with the curious observation that Friedman’s partial dependence plot has exactly the same…

用于估计与推断的深度神经网络AI 解读

We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second‐step inference after first‐step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric…

通过位移凸性分析两层神经网络AI 解读

Fitting a function by using linear combinations of a large number $N$ of “simple” components is one of the most fruitful ideas in statistical learning. This idea lies at the core of a variety of methods, from two-layer neural networks to kernel regression, to boosting. In general, the resulting risk minimization problem is nonconvex and is solved by gradient descent or its variants. Unfortunately, little is known about global convergence properties of these approaches. Here,…

带置信保证的交叉验证AI 解读

Cross-validation is one of the most popular model and tuning parameter selection methods in statistics and machine learning. Despite its wide applicability, traditional cross-validation methods tend to overfit, due to the ignorance of the uncertainty in the testing sample. We develop a novel statistically principled inference tool based on cross-validation that takes into account the uncertainty in the testing sample. This method outputs a set of highly competitive candidate…

深度 Knockoff 方法AI 解读

This article introduces a machine for sampling approximate model-X knockoffs for arbitrary and unspecified data distributions using deep generative models. The main idea is to iteratively refine a knockoff sampling mechanism until a criterion measuring the validity of the produced knockoffs is optimized; this criterion is inspired by the popular maximum mean discrepancy in machine learning and can be thought of as measuring the distance to pairwise exchangeability between…

社群识别中均值场变分推断的理论与计算保证AI 解读

The mean field variational Bayes method is becoming increasingly popular in statistics and machine learning. Its iterative coordinate ascent variational inference algorithm has been widely applied to large scale Bayesian inference. See Blei et al. (2017) for a recent comprehensive review. Despite the popularity of the mean field method, there exist remarkably little fundamental theoretical justifications. To the best of our knowledge, the iterative algorithm has never been…

可视化黑箱监督学习模型中预测变量的效应AI 解读

Summary In many supervised learning applications, understanding and visualizing the effects of the predictor variables on the predicted response is of paramount importance. A shortcoming of black box supervised learning models (e.g. complex trees, neural networks, boosted trees, random forests, nearest neighbours, local kernel-weighted methods and support vector regression) in this regard is their lack of interpretability or transparency. Partial dependence plots, which are…

深度神经网络与计量方法预测气候变化对农业产量影响的比较AI 解读

Summary Predicting the impact of climate change on crop yield is difficult, in part because the production function mapping weather to yield is high dimensional and nonlinear. We compare three approaches to predicting yields: (a) deep neural networks (DNNs), (b) traditional panel-data models, and (c) a new panel-data model that allows for unit and time fixed effects in both intercepts and slopes in the agricultural production function—made feasible by a new estimator called…

摘要暂缺

关于“使用 ReLU 深度神经网络进行非参数回归”的讨论AI 解读

First we would like to congratulate Professor Johannes Schmidt-Hieber for his excellent paper, which shows the surprising result that deep neural networks can achieve good rates of convergence even in case of nonsmooth activation functions.In the following we divide our discussion into three parts:1.The importance of compository assumptions.2. The necessity of the sparsity of the networks.3. The theoretical difference between ReLU and sigmoidal functions.

关于“使用 ReLU 深度神经网络进行非参数回归”的讨论AI 解读

I would like to congratulate Johannes Schmidt–Hieber on a very interesting paper in which he considers regression functions belonging to the class of so-called compositional functions and analyzes the ability of estimators based on the multivariate nonparametric regression model of deep neural networks to achieve minimax rates of convergence. In my discussion, I will first regard such a type of result from the general viewpoint of the theoretical foundations of deep neural…

关于“使用 ReLU 深度神经网络进行非参数回归”的讨论AI 解读

We congratulate Johannes Schmidt-Hieber for his elegant and thought-provoking results.His article uses deep-learning-inspired methods in the context of nonparametric regression.Schmidt-Hieber defines a rich class of composition-based functions G(q, d, t, β, K) and a class of sparse multilayer neural networks F(L, p, s, F ).He proves that least squares estimation over the class of sparse neural networks (with suitably chosen architecture (L, p, s, F )) achieves nearly minimax…

Mondrian 树与森林的极小极大最优速率AI 解读

Introduced by Breiman (Mach. Learn. 45 (2001) 5–32), Random Forests are widely used classification and regression algorithms. While being initially designed as batch algorithms, several variants have been proposed to handle online learning. One particular instance of such forests is the Mondrian forest (In Adv. Neural Inf. Process. Syst. (2014) 3140–3148; In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS) (2016)), whose…