论文检索

讨论：变量数远大于样本量时的 Dantzig 选择器统计估计AI 解读

Professors Candès and Tao are to be congratulated for their innovative and valuable contribution to high-dimensional sparse recovery and model selection.The analysis of vast data sets now commonly arising in scientific investigations poses many statistical challenges not present in smaller scale studies.Many of these data sets exhibit sparsity where most of the data corresponds to noise and only a small fraction is of interest.The needs of this research have excited much…

未知连接函数的一般非参数回归模型的速率最优估计AI 解读

This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that…

讨论：Lasso、L2Boosting 与 Dantzig 三种相关方法AI 解读

We would like to congratulate the authors for their thought-provoking and interesting paper.The Dantzig paper is on the timely topic of high-dimensional data modeling that has been the center of much research lately and where many exciting results have been obtained.It also falls in the very hot area at the interface of statistics and optimization: ℓ 1 -constrained minimization in linear models for computationally efficient model selection, or sparse model estimation (Chen,…

Dantzig 选择器：变量数远大于样本量时的统计估计AI 解读

In many important statistical applications, the number of variables or parameters p is much larger than the number of observations n. Suppose then that we have observations y=Xβ+z, where β∈Rp is a parameter vector of interest, X is a data matrix with possibly far fewer rows than columns, n≪p, and the zi’s are i.i.d. N(0, σ2). Is it possible to estimate β reliably based on the noisy data y? To estimate β, we introduce a new estimator—we call it the Dantzig selector—which is a…

相关误差下惩罚样条平滑的一则说明AI 解读

We investigate the behavior of data-driven smoothing parameters for penalized spline regression in the presence of correlated data. It has been shown for other smoothing methods that mean squared error minimizers, such as (generalized) cross-validation or the Akaike information criterion, are extremely sensitive to misspecifications of the correlation structure resulting in over- or (under-)fitting the data. In contrast to this, we show that a maximum likelihood–based choice…

Lasso 的有效自由度AI 解读

We study the effective degrees of freedom of the lasso in the framework of Stein’s unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate for the degrees of freedom of the lasso—a conclusion that requires no special assumption on the predictors. In addition, the unbiased estimator is shown to be asymptotically consistent. With these results on hand, various model selection criteria—Cp, AIC and BIC—are available, which, along…

贝叶斯非参数混合模型中强化机制的控制AI 解读

Summary The paper deals with the problem of determining the number of components in a mixture model. We take a Bayesian non-parametric approach and adopt a hierarchical model with a suitable non-parametric prior for the latent structure. A commonly used model for such a problem is the mixture of Dirichlet process model. Here, we replace the Dirichlet process with a more general non-parametric prior obtained from a generalized gamma process. The basic feature of this model is…

广义线性模型的 L1 正则化路径算法AI 解读

Summary We introduce a path following algorithm for L1-regularized generalized linear models. The L1-regularization procedure is useful especially because it, in effect, selects variables according to the amount of penalization on the L1-norm of the coefficients, in a manner that is less greedy than forward selection–backward deletion. The generalized linear model path algorithm efficiently computes solutions along the entire regularization path by using the…

函数型主成分回归与函数型偏最小二乘AI 解读

Regression of a scalar response on signal predictors, such as near-infrared (NIR) spectra of chemical samples, presents a major challenge when, as is typically the case, the dimension of the signals far exceeds their number. Most solutions to this problem reduce the dimension of the predictors either by regressing on components [e.g., principal component regression (PCR) and partial least squares (PLS)] or by smoothing methods, which restrict the coefficient function to the…

有限混合回归模型中的变量选择AI 解读

In the applications of finite mixture of regression (FMR) models, often many covariates are used, and their contributions to the response variable vary from one component to another of the mixture model. This creates a complex variable selection problem. Existing methods, such as the Akaike information criterion and the Bayes information criterion, are computationally expensive as the number of covariates and components in the mixture model increases. In this article we…

通过最小二乘近似统一 LASSO 估计AI 解读

We propose a method of least squares approximation (LSA) for unified yet simple LASSO estimation. Our general theoretical framework includes ordinary least squares, generalized linear models, quantile regression, and many others as special cases. Specifically, LSA can transfer many different types of LASSO objective functions into their asymptotically equivalent least squares problems. Thereafter, the standard asymptotic theory can be established and the LARS algorithm can…

最优地统计模型选择AI 解读

In many fields of science, predicting variables of interest over a study region based on noisy data observed at some locations is an important problem. Two popular methods for the problem are kriging and smoothing splines. The former assumes that the underlying process is stochastic, whereas the latter assumes it is purely deterministic. Kriging performs better than smoothing splines in some situations, but is outperformed by smoothing splines in others. However, little is…

用于发现基因调控网络的回归混合模型变量选择AI 解读

The profusion of genomic data through genome sequencing and gene expression microarray technology has facilitated statistical research in determining gene interactions regulating a biological process. Current methods generally consist of a two-stage procedure: clustering gene expression measurements and searching for regulatory “switches,” typically short, conserved sequence patterns (motifs) in the DNA sequence adjacent to the genes. This process often leads to misleading…

基于种群的可逆跳跃马尔可夫链蒙特卡洛AI 解读

In this paper we present an extension of population-based Markov chain Monte Carlo (MCMC) to the trans-dimensional case. One of the main challenges in MCMC-based inference is that of simulating from high and trans-dimensional target measures. In such cases, MCMC methods may not adequately traverse the support of the target; the simulation results will be unreliable. We develop population methods to deal with such problems, and give a result proving the uniform ergodicity of…

SCAD 方法的调节参数选择AI 解读

The penalised least squares approach with smoothly clipped absolute deviation penalty has been consistently demonstrated to be an attractive regression shrinkage and selection method. It not only automatically and consistently selects the important variables, but also produces estimators which are as efficient as the oracle estimator. However, these attractive features depend on appropriately choosing the tuning parameter. We show that the commonly used the generalised…

Cox 比例风险模型的自适应 LassoAI 解读

We investigate the variable selection problem for Cox's proportional hazards model, and propose a unified model selection and estimation procedure with desired theoretical properties and computational convenience. The new method is based on a penalized log partial likelihood with the adaptively weighted L1 penalty on regression coefficients, providing what we call the adaptive Lasso estimator. The method incorporates different penalties for different coefficients:…

Bayesian variable selection has gained much empirical success recently in a variety of applications when the number K of explanatory variables $(x_1,\ldots,x_K)$ is possibly much larger than the sample size $n$. For generalized linear models, if most of the $x_j$’s have very small effects on the response $y$, we show that it is possible to use Bayesian variable selection to reduce overfitting caused by the curse of dimensionality $K\gg n$. In this approach a suitable prior…

高斯回归中的聚合AI 解读

This paper studies statistical aggregation procedures in the regression setting. A motivating factor is the existence of many different methods of estimation, leading to possibly competing estimators. We consider here three different types of aggregation: model selection (MS) aggregation, convex (C) aggregation and linear (L) aggregation. The objective of (MS) is to select the optimal single estimator from the list; that of (C) is to select the optimal convex combination of…

一种适用于S估计的模型选择方法AI 解读

Cleaning data or removing some data periods in least squares (LS) regression analysis is not unusual. This practice indicates that a researcher sometimes desires to estimate the parameter value, with which the regression function fits a large fraction of individuals or events in the population (behind the original data set), possibly exhibiting poor fits to some atypical individuals or events. The S-estimators are a class of estimators that are consistent with the…

通过 LAD-lasso 实现稳健回归收缩与一致变量选择AI 解读

The least absolute deviation (LAD) regression is a useful method for robust regression, and the least absolute shrinkage and selection operator (lasso) is a popular choice for shrinkage estimation and variable selection. In this article we combine these two classical ideas together to produce LAD-lasso. Compared with the LAD regression, LAD-lasso can do parameter estimation and variable selection simultaneously. Compared with the traditional lasso, LAD-lasso is resistant to…

基于系数符号预测的模型选择方法AI 解读

Summary We consider a Bayesian model selection strategy based on predicting the signs of the coefficients in a regression model, i.e. we consider identification of coefficients in a full or encompassing model for which we can confidently predict whether they are positive or negative. This is useful when our main purpose in doing model selection is interpretation, since the sign of a coefficient is often of primary importance for this task. In the case of a linear model with…

多元线性回归中的降维与系数估计AI 解读

Summary We introduce a general formulation for dimension reduction and coefficient estimation in the multivariate linear model. We argue that many of the existing methods that are commonly used in practice can be formulated in this framework and have various restrictions. We continue to propose a new method that is more flexible and more generally applicable. The method proposed can be formulated as a novel penalized least squares estimate. The penalty that we employ is the…

L1范数多分类支持向量机AI 解读

Binary support vector machines (SVMs) have been proven to deliver high performance. In multiclass classification, however, issues remain with respect to variable selection. One challenging issue is classification and variable selection in the presence of variables in the magnitude of thousands, greatly exceeding the size of training sample. This often occurs in genomics classification. To meet the challenge, this article proposes a novel multiclass support vector machine,…

空间贝叶斯变量选择及其在功能磁共振成像中的应用AI 解读

We propose a procedure to undertake Bayesian variable selection and model averaging for a series of regressions located on a lattice. For those regressors that are in common in the regressions, we consider using an Ising prior to smooth spatially the indicator variables representing whether or not the variable is zero or nonzero in each regression. This smooths spatially the probabilities that each independent variable is nonzero in each regression and indirectly smooths…