论文检索

高维无岭最小二乘插值中的意外现象AI 解读

Interpolators—estimators that achieve zero training error—have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum ℓ2 norm (“ridgeless”) interpolation least squares regression, focusing on the high-dimensional regime in which the number of unknown parameters p is of the same order as the number of samples n. We consider two different models for the feature…

模型复杂度能否提升资产配置表现？机器学习预测模型的证据AI 解读

Summary This study evaluates the benefits of integrating return forecasts from a variety of machine learning and forecast combination methods into an out‐of‐sample asset allocation framework. The economic evaluation of the forecasts shows that model complexity translates to improved results in the majority of cases considered, with shrinkage methods and shallow neural networks generating the highest individual performance. Overall, an investor would consistently realize…

采用可逆 SGLD 提议的小批量 Metropolis–Hastings 方法AI 解读

Traditional Markov chain Monte Carlo (MCMC) algorithms are computationally intensive and do not scale well to large data. In particular, the Metropolis–Hastings (MH) algorithm requires passing over the entire dataset to evaluate the likelihood ratio in each iteration. We propose a general framework for performing MH-MCMC using mini-batches of the whole dataset and show that this gives rise to approximately a tempered stationary distribution. We prove that the algorithm…

嵌入学习AI 解读

Numerical embedding has become one standard technique for processing and analyzing unstructured data that cannot be expressed in a predefined fashion. It stores the main characteristics of data by mapping it onto a numerical vector. An embedding is often unsupervised and constructed by transfer learning from large-scale unannotated data. Given an embedding, a downstream learning method, referred to as a two-stage method, is applicable to unstructured data. In this article,…

平滑样条半参数密度模型AI 解读

Jiahui Yua, Jian Shib, Anna Liuc & Yuedong Wangd*a Department of Mathematics and Statistics, Boston University, Boston, MAb PayPal, San Jose, CAc Department of Mathematics and Statistics, University of Massachusetts, Amherst, MAd Department Statistics and Applied Probability, University of California, Santa Barbara, CAColor versions of one or more of the figures in the article can be found online at www.tandfonline.com/r/JASA.Supplementary materials for this article are…

高维数据下条件平均处理效应的估计AI 解读

Given the unconfoundedness assumption, we propose new nonparametric estimators for the reduced dimensional conditional average treatment effect (CATE) function. In the first stage, the nuisance functions necessary for identifying CATE are estimated by machine learning methods, allowing the number of covariates to be comparable to or larger than the sample size. The second stage consists of a low-dimensional local linear regression, reducing CATE to a function of the…

巴西失业与失业保险对犯罪的影响AI 解读

We investigate the impact of job loss on crime and the mitigating role of unemployment benefits, exploiting detailed individual‐level data linking employment careers, criminal records, and welfare registries for the universe of male workers in Brazil. The probability of committing crimes increases on average by 23% for workers displaced by mass layoffs, and by slightly less for their cohabiting sons. Using causal forests, we show that the effect is entirely driven by young…

局部稳健的半参数估计AI 解读

Many economic and causal parameters depend on nonparametric or high dimensional first steps. We give a general construction of locally robust/orthogonal moment functions for GMM, where first steps have no effect, locally, on average moment functions. Using these orthogonal moments reduces model selection and regularization bias, as is important in many applications, especially for machine learning first steps. Also, associated standard errors are robust to misspecification…

因果与结构效应的自动去偏机器学习AI 解读

Many causal and structural effects depend on regressions. Examples include policy effects, average derivatives, regression decompositions, average treatment effects, causal mediation, and parameters of economic structural models. The regressions may be high‐dimensional, making machine learning useful. Plugging machine learners into identifying equations can lead to poor inference due to bias from regularization and/or model selection. This paper gives automatic debiasing for…

半参数估计量的影响函数AI 解读

There are many economic parameters that depend on nonparametric first steps. Examples include games, dynamic discrete choice, average exact consumer surplus, and treatment effects. Often estimators of these parameters are asymptotically equivalent to a sample average of an object referred to as the influence function. The influence function is useful in local policy analysis, in evaluating local sensitivity of estimators, and constructing debiased machine learning…

非线性半参数模型的正则化正交机器学习AI 解读

Summary This paper proposes a Lasso-type estimator for a high-dimensional sparse parameter identified by a single index conditional moment restriction (CMR). In addition to this parameter, the moment function can also depend on a nuisance function, such as the propensity score or the conditional choice probability, which we estimate by modern machine learning tools. We first adjust the moment function so that the gradient of the future loss function is insensitive (formally,…

反事实与个体处理效应的共形推断AI 解读

Abstract Evaluating treatment effect heterogeneity widely informs treatment decision making. At the moment, much emphasis is placed on the estimation of the conditional average treatment effect via flexible machine learning algorithms. While these methods enjoy some theoretical appeal in terms of consistency and convergence rates, they generally perform poorly in terms of uncertainty quantification. This is troubling since assessing risk is crucial for reliable…

简约式因子扩展 VAR：利用稀疏性纳入具有经济含义的因子AI 解读

Summary Induced sparsity in the factor loading matrix identifies the factor basis, while rotational identification is obtained ex post by clustering methods closely related to machine learning. We extract meaningful economic concepts from a high‐dimensional data set, which together with observed variables follow an unrestricted, reduced‐form VAR process. Including a comprehensive set of economic concepts allows reliable, fundamental structural analysis, even of the factor…

文本选择AI 解读

Text data is ultra-high dimensional, which makes machine learning techniques indispensable for textual analysis. Text is often selected—journalists, speechwriters, and others craft messages to target their audiences’ limited attention. We develop an economically motivated high-dimensional selection model that improves learning from text (and from sparse counts data more generally). Our model is especially useful when the choice to include a phrase is more interesting than…

用最优决策树自适应估计多元分段多项式与有界变差函数AI 解读

Proposed by Donoho (Ann. Statist. 25 (1997) 1870–1911), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article, we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the context of estimating piecewise smooth functions in general dimensions in the fixed design setup. More precisely, these optimal…

时变参数高维预测回归的 Boosting 估计AI 解读

High dimensional predictive regressions are useful in wide range of applications. However, the theory is mainly developed assuming that the model is stationary with time invariant parameters. This is at odds with the prevalent evidence for parameter instability in economic time series, but theories for parameter instability are mainly developed for models with a small number of covariates. In this paper, we present two L2 boosting algorithms for estimating high dimensional…

通过预测变量增广确定降维阶数AI 解读

Summary In many dimension reduction problems in statistics and machine learning, such as in principal component analysis, canonical correlation analysis, independent component analysis and sufficient dimension reduction, it is important to determine the dimension of the reduced predictor, which often amounts to estimating the rank of a matrix. This problem is called order determination. In this article, we propose a novel and highly effective order-determination method based…

用于α散度最小化的无限维梯度下降AI 解读

This paper introduces the (α,Γ)-descent, an iterative algorithm which operates on measures and performs α-divergence minimisation in a Bayesian framework. This gradient-based procedure extends the commonly-used variational approximation by adding a prior on the variational parameters in the form of a measure. We prove that for a rich family of functions Γ, this algorithm leads at each step to a systematic decrease in the α-divergence and derive convergence results. Our…

含时变协变量的非参数风险函数提升估计AI 解读

Given functional data from a survival process with time-dependent covariates, we derive a smooth convex representation for its nonparametric log-likelihood functional and obtain its functional gradient. From this we devise a generic gradient boosting procedure for estimating the hazard function nonparametrically. An illustrative implementation of the procedure using regression trees is described to show how to recover the unknown hazard. The generic estimator is consistent…

全连接深度神经网络回归估计的收敛速率AI 解读

Recent results in nonparametric regression show that deep learning, that is, neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper, we show that we can get similar results…

利用因果推断与机器学习评估1990年《清洁空气法》修正案的健康影响AI 解读

We develop a causal inference approach to estimate the number of adverse health events that were prevented due to changes in exposure to multiple pollutants attributable to a large-scale air quality intervention/regulation, with a focus on the 1990 Clean Air Act Amendments (CAAA). We introduce a causal estimand called the Total Events Avoided (TEA) by the regulation, defined as the difference in the number of health events expected under the no-regulation pollution exposures…

神经网络训练作为数据自适应核学习：表示与逼近收益AI 解读

Consider the problem: given the data pair (x,y) drawn from a population with f*(x)=E[y|x=x], specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does ft , the function computed by the neural network at time t, relate to f*, in terms of approximation and representation? What are the provable benefits of the adaptive representation by neural networks compared to the pre-specified fixed basis representation in the…

ALMOND：基于神经网络与 Langevin 扩散的自适应潜变量建模和优化AI 解读

Latent variable models cover a broad range of statistical and machine learning models, such as Bayesian models, linear mixed models, and Gaussian mixture models. Existing methods often suffer from two major challenges in practice: (a) a proper latent variable distribution is difficult to be specified; (b) making an exact likelihood inference is formidable due to the intractable computation. We propose a novel framework for the inference of latent variable models that…

AMF：用于在线学习的聚合 Mondrian 森林AI 解读

Abstract Random forest (RF) is one of the algorithms of choice in many supervised learning applications, be it classification or regression. The appeal of such tree-ensemble methods comes from a combination of several characteristics: a remarkable accuracy in a variety of tasks, a small number of parameters to tune, robustness with respect to features scaling, a reasonable computational cost for training and prediction, and their suitability in high-dimensional settings. The…