论文检索

以正则化最小二乘统一模型选择与稀疏恢复AI 解读

Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regularized least squares estimator enjoys a nonasymptotic property, called the weak oracle property, where the dimensionality can grow exponentially…

用于分组与层级变量选择的复合绝对惩罚族AI 解读

Extracting useful information from high-dimensional data is an important focus of today’s statistical research and practice. Penalized loss function minimization has been shown to be effective for this task both theoretically and empirically. With the virtues of both regularization and sparsity, the L1-penalized squared error minimization method Lasso has been popular in regression models and beyond. In this paper, we combine different norms including L1 to form an…

使用平滑自适应高斯混合的回归密度估计AI 解读

We model a regression density flexibly so that at each value of the covariates the density is a mixture of normals with the means, variances and mixture probabilities of the components changing smoothly as a function of the covariates. The model extends the existing models in two important ways. First, the components are allowed to be heteroscedastic regressions as the standard model with homoscedastic regressions can give a poor fit to heteroscedastic data, especially when…

结合变量选择的非参数贝叶斯条件分布建模AI 解读

This article considers a methodology for flexibly characterizing the relationship between a response and multiple predictors. Goals are (1) to estimate the conditional response distribution addressing the distributional changes across the predictor space, and (2) to identify important predictors for the response distribution change both within local regions and globally. We first introduce the probit stick-breaking process (PSBP) as a prior for an uncountable collection of…

基于 LASSO 的多变量统计过程控制AI 解读

This article develops a new multivariate statistical process control (SPC) methodology based on adapting the LASSO variable selection method to the SPC problem. The LASSO method has the sparsity property of being able to select exactly the set of nonzero regression coefficients in multivariate regression modeling, which is especially useful in cases where the number of nonzero coefficients is small. In multivariate SPC applications, process mean vectors often shift in a…

利用双向正则化奇异值分解分析双向函数型数据AI 解读

Two-way functional data consist of a data matrix whose row and column domains are both structured, for example, temporally or spatially, as when the data are time series collected at different locations in space. We extend one-way functional principal component analysis (PCA) to two-way functional data by introducing regularization of both left and right singular vectors in the singular value decomposition (SVD) of the data matrix. We focus on a penalization approach and…

超高维变量筛选中的前向回归AI 解读

Motivated by the seminal theory of Sure Independence Screening (Fan and Lv 2008, SIS), we investigate here another popular and classical variable screening method, namely, forward regression (FR). Our theoretical analysis reveals that FR can identify all relevant predictors consistently, even if the predictor dimension is substantially larger than the sample size. In particular, if the dimension of the true model is finite, FR can discover all relevant predictors within a…

高维回归中的 p 值AI 解读

Assigning significance in high-dimensional regression is challenging. Most computationally efficient selection algorithms cannot guard against inclusion of noise variables. Asymptotically valid p-values are not available. An exception is a recent proposal by Wasserman and Roeder that splits the data into two parts. The number of variables is then reduced to a manageable size using the first split, while classical variable selection techniques can be applied to the remaining…

广义非线性 ℓ1 正则化解路径的高效全局近似及应用AI 解读

We consider efficient construction of nonlinear solution paths for general ℓ1-regularization. Unlike the existing methods that incrementally build the solution path through a combination of local linear approximation and recalibration, we propose an efficient global approximation to the whole solution path. With the loss function approximated by a quadratic spline, we show that the solution path can be computed using a generalized Lars algorithm. The proposed methodology…

稀疏加性模型AI 解读

Summary We present a new class of methods for high dimensional non-parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non-parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin.…

融合 Lasso 的性质与改进AI 解读

We consider estimating an unknown signal, both blocky and sparse, which is corrupted by additive noise. We study three interrelated least squares procedures and their asymptotic properties. The first procedure is the fused lasso, put forward by Friedman et al. [Ann. Appl. Statist. 1 (2007) 302–332], which we modify into a different estimator, called the fused adaptive lasso, with better properties. The other two estimators we discuss solve least squares problems on sieves;…

非参数回归、置信区域与正则化AI 解读

In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and nonasymptotic confidence region $\mathcal{A}_{n}$ which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically center on certain simplest functions in $\mathcal{A}_{n}$ where simplicity can be defined in terms of shape (number of local extremes, intervals of…

L1 正则化最小二乘回归的若干精确性能界AI 解读

We derive sharp performance bounds for least squares regression with L1 regularization from parameter estimation accuracy and feature selection quality perspectives. The main result proved for L1 regularization extends a similar result in [Ann. Statist. 35 (2007) 2313–2351] for the Dantzig selector. It gives an affirmative answer to an open question in [Ann. Statist. 35 (2007) 2358–2364]. Moreover, the result leads to an extended view of feature selection that allows less…

通过 ℓ1 最小化实现近理想模型选择AI 解读

We consider the fundamental problem of estimating the mean of a vector y=Xβ+z, where X is an n×p design matrix in which one can have far more variables than observations, and z is a stochastic error term—the so-called “p>n” setup. When β is sparse, or, more generally, when there is a sparse subset of covariates providing a close approximation to the unknown mean vector, we ask whether or not it is possible to accurately estimate Xβ using a computationally tractable…

高维变量选择AI 解读

This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we select one model by cross-validation. In the third stage we use hypothesis testing to eliminate some variables. We refer to the first two stages as "screening"…

可解释的函数型线性回归AI 解读

Regression models to relate a scalar Y to a functional predictor X(t) are becoming increasingly common. Work in this area has concentrated on estimating a coefficient function, β(t), with Y related to X(t) through ∫β(t)X(t) dt. Regions where β(t)≠0 correspond to places where there is a relationship between X(t) and Y. Alternatively, points where β(t)=0 indicate no relationship. Hence, for interpretation purposes, it is desirable for a regression procedure to be capable of…

利用完整解曲面进行自适应正则化AI 解读

Several sparseness penalties have been suggested for delivery of good predictive performance in automatic variable selection within the framework of regularization. All assume that the true model is sparse. We propose a penalty, a convex combination of the L1- and L∞-norms, that adapts to a variety of situations including sparseness and nonsparseness, grouping and nongrouping. The proposed penalty performs grouping and adaptive regularization. In addition, we introduce a…

惩罚样条估计量的渐近性质AI 解读

We study the class of penalized spline estimators, which enjoy similarities to both regression splines, without penalty and with fewer knots than data points, and smoothing splines, with knots equal to the data points and a penalty controlling the roughness of the fit. Depending on the number of knots, sample size and penalty, we show that the theoretical properties of penalized regression spline estimators are either similar to those of regression splines or to those of…

利用倾斜方法评估分类器中各分量的影响AI 解读

Summary Many contemporary classifiers are constructed to provide good performance for very high dimensional data. However, an issue that is at least as important as good classification is determining which of the many potential variables provide key information for good decisions. Responding to this issue can help us to determine which aspects of the datagenerating mechanism (e.g. which genes in a genomic study) are of greatest importance in terms of distinguishing between…

残差可能不光滑的半参数条件矩模型的有效估计AI 解读

This paper considers semiparametric efficient estimation of conditional moment models with possibly nonsmooth residuals in unknown parametric components ([theta]) and unknown functions (h) of endogenous variables. We show that: (1) the penalized sieve minimum distance (PSMD) estimator can simultaneously achieve root-n asymptotic normality of and nonparametric optimal convergence rate of , allowing for noncompact function parameter spaces; (2) a simple weighted bootstrap…

含测量误差协变量平滑效应的 Cox 模型AI 解读

We propose, develop, and implement a fully Bayesian inferential approach for the Cox model when the log hazard function contains unknown smooth functions of the variables measured with error. Our approach is to model nonparametrically both the log-baseline hazard and the smooth components of the log-hazard functions using low-rank penalized splines. Careful implementation of the Bayesian inferential machinery is shown to produce remarkably better results than the naive…

大规模预测问题的经验贝叶斯估计AI 解读

Classical prediction methods such as Fisher's linear discriminant function were designed for small-scale problems, where the number of predictors N is much smaller than the number of observations n. Modern scientific devices often reverse this situation. A microarray analysis, for example, might include n = 100 subjects measured on N = 10,000 genes, each of which is a potential predictor. This paper proposes an empirical Bayes approach to large-scale prediction, where the…

参数维度随样本量增长时的自适应弹性网AI 解读

We consider the problem of model selection and estimation in situations where the number of parameters diverges with the sample size. When the dimension is high, an ideal method should have the oracle property (Fan and Li, 2001; Fan and Peng, 2004) which ensures the optimal large sample performance. Furthermore, the high-dimensionality often induces the collinearity problem which should be properly handled by the ideal method. Many existing variable selection methods fail to…

Lasso 与 Dantzig selector 的统一分析AI 解读

We show that, under a sparsity scenario, the Lasso estimator and the Dantzig selector exhibit similar behavior. For both methods, we derive, in parallel, oracle inequalities for the prediction risk in the general nonparametric regression model, as well as bounds on the ℓp estimation loss for 1≤p≤2 in the linear model when the number of variables can be much larger than the sample size.