论文检索

L2 Boosting 的性质刻画AI 解读

We consider $L_{2}$Boosting, a special case of Friedman’s generic boosting algorithm applied to linear regression under $L_{2}$-loss. We study $L_{2}$Boosting for an arbitrary regularization parameter and derive an exact closed form expression for the number of steps taken along a fixed coordinate direction. This relationship is used to describe $L_{2}$Boosting’s solution path, to describe new tools for studying its path, and to characterize some of the algorithm’s unique…

利用复杂抽样数据构建一致的回归树AI 解读

In the past several years a wide range of methods for the construction of regression trees and other estimators based on the recursive partitioning of samples have appeared in the statistics literature. Many applications involve data collected through a complex sample design. At present, however, relatively little is known regarding the properties of these methods under complex designs. This article proposes a method for incorporating information about the complex sample…

处理效应估计的因果图及其在有效协变量选择中的应用AI 解读

Careful examination of the structure determining treatment choice and outcomes, as advocated by Heckman (2008), is central to the design of treatment effect estimators and, in particular, proper choice of covariates. Here, we demonstrate how causal diagrams developed in the machine learning literature by Judea Pearl and his colleagues, but not so well known to economists, can play a key role in this examination by using these methods to give a detailed analysis of the choice…

利用贝叶斯降秩多元模型预测大型数据集AI 解读

The paper addresses the issue of forecasting a large set of variables using multivariate models. In particular, we propose three alternative reduced rank forecasting models and compare their predictive performance for US time series with the most promising existing alternatives, namely, factor models, large-scale Bayesian VARs, and multivariate boosting. Specifically, we focus on classical reduced rank regression, a two-step procedure that applies, in turn, shrinkage and…

用提升法加性分位数回归识别儿童严重营养不良风险因素AI 解读

We investigated the risk factors for childhood malnutrition in India based on the 2005/2006 Demographic and Health Survey by applying a novel estimation technique for additive quantile regression. Ordinary linear and generalized linear regression models relate the mean of a response variable to a linear combination of covariate effects, and, as a consequence, focus on average properties of the response. The use of such a regression model for analyzing childhood malnutrition…

EEBoost：基于估计方程的预测与变量选择AI 解读

The modern statistical literature is replete with methods for performing variable selection and prediction in standard regression problems. However, simple models may misspecify or fail to capture important aspects of the data generating process such as missingness, correlation, and over/underdispersion. In this article we describe EEBoost, a strategy for variable selection and prediction which can be applied in high-dimensional settings where inference for low-dimensional…

大型数据集中的快速稳健模型选择AI 解读

Large datasets are increasingly common in many research fields. In particular, in the linear regression context, it is often the case that a huge number of potential covariates are available to explain a response variable, and the first step of a reasonable statistical analysis is to reduce the number of covariates. This can be done in a forward selection procedure that includes selecting the variable to enter, deciding to retain it or stop the selection, and estimating the…

用于学习与试验设计的动态树AI 解读

Dynamic regression trees are an attractive option for automatic regression and classification with complicated response surfaces in online application settings. We create a sequential tree model whose state changes in time with the accumulation of new data, and provide particle learning algorithms that allow for the efficient online posterior filtering of tree states. A major advantage of tree regression is that it allows for the use of very simple models within each…

分层马尔可夫正态混合模型及金融资产收益应用AI 解读

Motivated by the common problem of constructing predictive distributions for daily asset returns over horizons of one to several trading days, this article introduces a new model for time series. This model is a generalization of the Markov normal mixture model in which the mixture components are themselves normal mixtures, and it is a specific case of an artificial neural network model with two hidden layers. The article uses the model to construct predictive distributions…

函数型预测变量的最具预测力设计点AI 解读

We suggest a way of reducing the very high dimension of a functional predictor, X, to a low number of dimensions chosen so as to give the best predictive performance. Specifically, if X is observed on a fine grid of design points t1,…, tr, we propose a method for choosing a small subset of these, say ti1,…, tik, to optimize the prediction of a response variable, Y. The values tij are referred to as the most predictive design points, or covariates, for a given value of k, and…

基于 Boosting 与径向基神经网络近似的鞅差假设检验AI 解读

The martingale difference restriction is an outcome of many theoretical analyses in economics and finance. A large body of econometric literature deals with tests of that restriction. We provide new tests based on radial basis function (RBF) neural networks. Our work is based on the test design of Blake and Kapetanios (2000, 2003a, 2003b). However, unlike that work we provide a formal theoretical justification for the validity of these tests and present some new general…

用于高维回归与分类的惩罚 Bregman 散度AI 解读

Regularization methods are characterized by loss functions measuring data fits and penalty terms constraining model parameters. The commonly used quadratic loss is not suitable for classification with binary responses, whereas the loglikelihood function is not readily applicable to models where the exact distribution of observations is unknown or not fully specified. We introduce the penalized Bregman divergence by replacing the negative loglikelihood in the conventional…

生存数据的高维变量选择AI 解读

The minimal depth of a maximal subtree is a dimensionless order statistic measuring the predictiveness of a variable in a survival tree. We derive the distribution of the minimal depth and use it for high-dimensional variable selection using random survival forests. In big p and small n problems (where p is the dimension and n is the sample size), the distribution of the minimal depth reveals a “ceiling effect” in which a tree simply cannot be grown deep enough to properly…

金融波动率的样条建模AI 解读

Summary We propose a flexible generalized auto-regressive conditional heteroscedasticity type of model for the prediction of volatility in financial time series. The approach relies on the idea of using multivariate B-splines of lagged observations and volatilities. Estimation of such a B-spline basis expansion is constructed within the likelihood framework for non-Gaussian observations. As the dimension of the B-spline basis is large, i.e. many parameters, we use…

在线预测线性回归AI 解读

We consider the on-line predictive version of the standard problem of linear regression; the goal is to predict each consecutive response given the corresponding explanatory variables and all the previous observations. The standard treatment of prediction in linear regression analysis has two drawbacks: (1) the classical prediction intervals guarantee that the probability of error is equal to the nominal significance level ɛ, but this property per se does not imply that the…

扩散指数预测中的提升法AI 解读

Abstract In forecasting and regression analysis, it is often necessary to select predictors from a large feasible set. When the predictors have no natural ordering, an exhaustive evaluation of all possible combinations of the predictors can be computationally costly. This paper considers ‘boosting’ as a methodology of selecting the predictors in factor‐augmented autoregressions. As some of the predictors are being estimated, we propose a stopping rule for boosting to prevent…

非参数变量选择：EARTH算法AI 解读

We consider regression experiments involving a response variable Y and a large number of predictor variables X1, …, Xd, many of which may be irrelevant for the prediction of Y and thus must be removed before Y can be predicted from the X’s. We consider two procedures that select variables by using importance scores that measure the strength of the relationship between predictor variables and a response and keep those variables whose importance scores exceed a threshold. In…

主成分分析的有限样本近似：矩阵扰动方法AI 解读

Principal component analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size n, and those of the limiting population PCA as n→∞. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness…

具有异方差误差的神经网络需求系统AI 解读

In this paper we consider estimation of demand systems with flexible functional forms, allowing an error term with a general conditional heteroskedasticity function that depends on observed covariates, such as demographic variables. We propose a general model that can be estimated either by quasi-maximum likelihood (in the case of exogenous regressors) or generalized method of moments (GMM) if the covariates are endogenous. The specification proposed in the paper nests…

需求估计的另一种方法：含条件波动的高频航空旅客到达量神经网络回归AI 解读

In this paper we provide an alternative approach to analyze the demand for international tourism in the Balearic Islands, Spain, by using a neural network model that incorporates time-varying conditional volatility. We consider daily air passenger arrivals to Palma de Mallorca, Ibiza and Mahon, which are located in the islands of Mallorca, Ibiza and Menorca, respectively, as a proxy for international tourism demand for the Balearic Islands. Spain is a world leader in terms…

高维线性回归中 Lasso 变量选择的稀疏性与偏差AI 解读

Meinshausen and Buhlmann [Ann. Statist. 34 (2006) 1436–1462] showed that, for neighborhood selection in Gaussian graphical models, under a neighborhood stability condition, the LASSO is consistent, even when the number of variables is of greater order than the sample size. Zhao and Yu [(2006) J. Machine Learning Research 7 2541–2567] formalized the neighborhood stability condition in the context of linear regression as a strong irrepresentable condition. That paper showed…

机器学习中的核方法AI 解读

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as…

排序与 U 统计量的经验最小化AI 解读

The problem of ranking/ordering instances, instead of simply classifying them, has recently gained much attention in machine learning. In this paper we formulate the ranking problem in a rigorous statistical framework. The goal is to learn a ranking rule for deciding, among two instances, which one is “better,” with minimum ranking risk. Since the natural estimates of the risk are of the form of a U-statistic, results of the theory of U-processes are required for…

利用平滑间隔函数分析提升算法AI 解读

We introduce a useful tool for analyzing boosting algorithms called the “smooth margin function,” a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, “coordinate ascent boosting” and “approximate coordinate ascent boosting,” which are similar to Freund and Schapire’s AdaBoost algorithm and Breiman’s arc-gv algorithm. We give convergence rates to the maximum margin solution for both of our…