论文检索

Lasso 型广义矩估计量AI 解读

This paper proposes the least absolute shrinkage and selection operator–type (Lasso-type) generalized method of moments (GMM) estimator. This Lasso-type estimator is formed by the GMM objective function with the addition of a penalty term. The exponent of the penalty term in the regular Lasso estimator is equal to one. However, the exponent of the penalty term in the Lasso-type estimator is less than one in the analysis here. The magnitude of the exponent is reduced to avoid…

用于无模型变量选择的收缩逆回归估计AI 解读

Summary The family of inverse regression estimators that was recently proposed by Cook and Ni has proven effective in dimension reduction by transforming the high dimensional predictor vector to its low dimensional projections. We propose a general shrinkage estimation strategy for the entire inverse regression estimation family that is capable of simultaneous dimension reduction and variable selection. We demonstrate that the new estimators achieve consistency in variable…

DASSO：Dantzig 选择器与 Lasso 的联系AI 解读

Summary We propose a new algorithm, DASSO, for fitting the entire coefficient path of the Dantzig selector with a similar computational cost to the least angle regression algorithm that is used to compute the lasso. DASSO efficiently constructs a piecewise linear path through a sequential simplex-like algorithm, which is remarkably similar to the least angle regression algorithm. Comparison of the two algorithms sheds new light on the question of how the lasso and Dantzig…

采用非光滑惩罚的有限混合模型阶数选择AI 解读

Order selection is a fundamental and challenging problem in the application of finite mixture models. We develop a new penalized likelihood approach that we call MSCAD. MSCAD deviates from information-based methods, such as Akaike information criterion and the Bayes information criterion, by introducing two penalty functions that depend on the mixing proportions and the component parameters. It is consistent in estimating both the order of the mixture model and the mixing…

高维情形下的平滑截断绝对偏差估计AI 解读

The smoothly clipped absolute deviation (SCAD) estimator, proposed by Fan and Li, has many desirable properties, including continuity, sparsity, and unbiasedness. The SCAD estimator also has the (asymptotically) oracle property when the dimension of covariates is fixed or diverges more slowly than the sample size. In this article we study the SCAD estimator in high-dimensional settings where the dimension of covariates can be much larger than the sample size. First, we…

二元结局预测中变量选择的随机优化比较及卫生政策应用AI 解读

Traditional variable-selection strategies in generalized linear models (GLMs) seek to optimize a measure of predictive accuracy without regard for the cost of data collection. When the purpose of such model building is the creation of predictive scales to be used in future studies with constrained budgets, the standard approach may not be optimal. We propose a Bayesian decision-theoretic framework for variable selection in binary-outcome GLMs where the budget for data…

非参数变量选择：EARTH算法AI 解读

We consider regression experiments involving a response variable Y and a large number of predictor variables X1, …, Xd, many of which may be irrelevant for the prediction of Y and thus must be removed before Y can be predicted from the X’s. We consider two procedures that select variables by using importance scores that measure the strength of the relationship between predictor variables and a response and keep those variables whose importance scores exceed a threshold. In…

重复测量分析中非参数变系数模型的变量选择AI 解读

Nonparametric varying-coefficient models are commonly used for analysis of data measured repeatedly over time, including longitudinal and functional responses data. While many procedures have been developed for estimating the varying-coefficients, the problem of variable selection for such models has not been addressed. In this article, we present a regularized estimation procedure for variable selection that combines basis function approximations and the smoothly clipped…

二项回归模型中 Jeffreys 先验的性质与计算实现AI 解读

We study several theoretical properties of Jeffreys's prior for binomial regression models. We show that Jeffreys's prior is symmetric and unimodal for a class of binomial regression models. We characterize the tail behavior of Jeffreys's prior by comparing it with the multivariate t and normal distributions under the commonly used logistic, probit, and complementary log-log regression models. We also show that the prior and posterior normalizing constants under Jeffreys's…

函数型加性模型AI 解读

In commonly used functional regression models, the regression of a scalar or functional response on the functional predictor is assumed to be linear. This means that the response is a linear function of the functional principal component scores of the predictor process. We relax the linearity assumption and propose to replace it by an additive structure, leading to a more widely applicable and much more flexible framework for functional regression models. The proposed…

随机矩阵理论：SAMSI 研究计划介绍AI 解读

Craig Tracy and I became coorganizers and the program's scope was enlarged.It now included regularization issues, inference in graphical models, Bayesian multivariate analysis in such models, applications of random matrix theory particularly in electrical engineering, as well as applications of regularization and high dimensional multivariate analysis in geophysical models.The program was realized as a six-month session in 2006-2007 and featured not only the usual opening…

正则化样本协方差矩阵的中心极限定理AI 解读

We consider the spectral properties of a class of regularized estimators of (large) empirical covariance matrices corresponding to stationary (but not necessarily Gaussian) sequences, obtained by banding. We prove a law of large numbers (similar to that proved in the Gaussian case by Bickel and Levina), which implies that the spectrum of a banded empirical covariance matrix is an efficient estimator. Our main result is a central limit theorem in the same regime, which to our…

通过阈值化实现协方差正则化AI 解读

This paper considers regularizing a covariance matrix of p variables estimated from n observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and (log p)/n→0, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an…

弹性估计的推断方法AI 解读

Elasticities are often estimated from the results of demand analysis. However, drawing inferences from them may involve assumptions that could influence the outcome. In this paper we investigate one of the most common forms of elasticity which is defined as a ratio of estimated relationships and demonstrate how the Fieller method for the construction of confidence intervals can be used to draw inferences. We estimate the elasticities of expenditure from Engel curves using a…

超高维特征空间的确定性独立筛选AI 解读

Summary Variable selection plays an important role in high dimensional statistical modelling which nowadays appears in many areas and is key to various scientific discoveries. For problems of large scale or dimensionality p, accuracy of estimation and computational cost are two top concerns. Recently, Candes and Tao have proposed the Dantzig selector using L1-regularization and showed that it achieves the ideal risk up to a logarithmic factor log(p). Their innovative…

高维分类与数据挖掘中的 Gibbs 后验变量选择AI 解读

In the popular approach of “Bayesian variable selection” (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is constructed from a risk function of practical interest (such as the classification error) and aims at minimizing a risk function without modeling the data…

利用主成分联合建模成对稀疏函数数据AI 解读

We propose a modelling framework to study the relationship between two paired longitudinally observed variables. The data for each variable are viewed as smooth curves measured at discrete time-points plus random errors. While the curves for each variable are summarized using a few important principal components, the association of the two longitudinal variables is modelled through the association of the principal component scores. We use penalized splines to model the mean…

大模型空间下用于模型选择的扩展贝叶斯信息准则AI 解读

The ordinary Bayesian information criterion is too liberal for model selection when the model space is large. In this paper, we re-examine the Bayesian paradigm for model selection and propose an extended family of Bayesian information criteria, which take into account both the number of unknown parameters and the complexity of the model space. Their consistency is established, in particular allowing the number of covariates to increase to infinity with the sample size.…

变量纳入与收缩算法AI 解读

The Lasso is a popular and computationally efficient procedure for automatically performing both variable selection and coefficient shrinkage on linear regression models. One limitation of the Lasso is that the same tuning parameter is used for both variable selection and shrinkage. As a result, it typically ends up selecting a model with too many variables to prevent overshrinkage of the regression coefficients. We suggest an improved class of methods called variable…

混合模型选择的围栏方法AI 解读

Many model search strategies involve trading off model fit with model complexity in a penalized goodness of fit measure. Asymptotic properties for these types of procedures in settings like linear regression and ARMA time series have been studied, but these do not naturally extend to nonstandard situations such as mixed effects models, where simple definition of the sample size is not meaningful. This paper introduces a new class of strategies, known as fence methods, for…

高维变量选择与回归中的“预处理”AI 解读

We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a “preconditioned” response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the preconditioned response variable. In a number of simulated…

高维线性回归中 Lasso 变量选择的稀疏性与偏差AI 解读

Meinshausen and Buhlmann [Ann. Statist. 34 (2006) 1436–1462] showed that, for neighborhood selection in Gaussian graphical models, under a neighborhood stability condition, the LASSO is consistent, even when the number of variables is of greater order than the sample size. Zhao and Yu [(2006) J. Machine Learning Research 7 2541–2567] formalized the neighborhood stability condition in the context of linear regression as a strong irrepresentable condition. That paper showed…

非凹惩罚似然模型中的一步稀疏估计AI 解读

Fan & Li (2001) propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective function is nondifferentiable and nonconcave. In this article we propose a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized…

异方差测量误差下的密度估计AI 解读

We consider density estimation when the variable of interest is subject to heteroscedastic measurement error. The density is assumed to have a smooth but unknown functional form that we model with a penalized mixture of B-splines. We treat the situation in which multiple mismeasured observations of each variable of interest are observed for at least some of the subjects, and the measurement error is assumed to be additive and normal. The measurement error variance function…