论文检索

高维广义线性模型与 LassoAI 解读

We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.

高维加性模型中半参数函数的贝叶斯识别、选择与估计AI 解读

In this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the…

估计误差的快速收敛率与模型选择的 Oracle 不等式AI 解读

We consider complexity penalization methods for model selection. These methods aim to choose a model to optimally trade off estimation and approximation errors by minimizing the sum of an empirical risk term and a complexity penalty. It is well known that if we use a bound on the maximal deviation between empirical and true risks as a complexity penalty, then the risk of our choice is no more than the approximation error plus twice the complexity penalty. There are many…

回归中的局部化模型选择AI 解读

Research on model/procedure selection has focused on selecting a single model globally. In many applications, especially for high-dimensional or complex data, however, the relative performance of the candidate procedures typically depends on the location, and the globally best procedure can often be improved when selection of a model is allowed to depend on location. We consider localized model selection methods and derive their theoretical properties.This research was…

最高后验概率模型选择的深入考察AI 解读

We consider the properties of the highest posterior probability model in a linear regression setting. Under a spike and slab hierarchy we find that although highest posterior model selection is total risk consistent, it possesses hidden undesirable properties. One such property is a marked underfitting in finite samples, a phenomenon well noted for Bayesian information criterion (BIC) related procedures but not often associated with highest posterior model selection. Another…

近奇异设计下的收缩估计AI 解读

Shrinkage estimation procedures such as ridge regression and the lasso have been proposed for stabilizing estimation in linear models when high collinearity exists in the design. In this paper, we consider asymptotic properties of shrinkage estimators in the case of “nearly singular” designs.I thank Hannes Leeb and Benedikt Pötscher and also the referees for their valuable comments. This research was supported by a grant from the Natural Sciences and Engineering Research…

相关与协方差矩阵的惩罚正态似然和岭正则化AI 解读

High dimensionality causes problems in various areas of statistics. A particular situation that rarely has been considered is the testing of hypotheses about multivariate regression models in which the dimension of the multivariate response is large. In this article a ridge regularization approach is proposed in which either the covariance or the correlation matrix is regularized to ensure nonsingularity irrespective of the dimensionality of the data. It is shown that the…

用于贝叶斯变量选择的 g 先验混合AI 解读

Zellner's g prior remains a popular conventional prior for use in Bayesian variable selection, despite several undesirable consistency issues. In this article we study mixtures of g priors as an alternative to default g priors that resolve many of the problems with the original formulation while maintaining the computational tractability that has made the g prior so popular. We present theoretical properties of the mixture g priors and provide real and simulated examples to…

贝叶斯层级曲线配准AI 解读

Functional data often exhibit a common shape, but with variations in amplitude and phase across curves. The analysis often proceeds by synchronization of the data through curve registration. In this article we propose a Bayesian hierarchical model for curve registration. Our hierarchical model provides a formal account of amplitude and phase variability while borrowing strength from the data across curves in the estimation of the model parameters. We discuss extensions of…

通过共同因子建模多元时间序列AI 解读

We propose a new method for estimating common factors of multiple time series. One distinctive feature of the new approach is that it is applicable to some nonstationary time series. The unobservable, nonstationary factors are identified by expanding the white noise space step by step, thereby solving a high-dimensional optimization problem by several low-dimensional sub-problems. Asymptotic properties of the estimation are investigated. The proposed methodology is…

惩罚样条的渐近性质AI 解读

We study the asymptotic behaviour of penalized spline estimators in the univariate case. We use B-splines and a penalty is placed on mth-order differences of the coefficients. The number of knots is assumed to converge to infinity as the sample size increases. We show that penalized splines behave similarly to Nadaraya--Watson kernel estimators with ‘equivalent’ kernels depending upon m. The equivalent kernels we obtain for penalized splines are the same as those found by…

变量重要性的层级检验AI 解读

A frequently encountered challenge in high-dimensional regression is the detection of relevant variables. Variable selection suffers from instability and the power to detect relevant variables is typically low if predictor variables are highly correlated. When taking the multiplicity of the testing problem into account, the power diminishes even further. To gain power and insight, it can be advantageous to look for influence not at the level of individual variables but…

高维空间中参数回归模型的诊断AI 解读

We study tools for checking the validity of a parametric regression model. When the dimension of the regressors is large, many of the existing tests face the curse of dimensionality or require some ordering of the data. Our tests are based on the residual empirical process marked by proper functions of the regressors. They are able to detect local alternatives converging to the null at parametric rates. Parametric and nonparametric alternatives are considered. In the latter…

利用惩罚样条回归进行非参数小区域估计AI 解读

Summary The paper proposes a small area estimation approach that combines small area random effects with a smooth, non-parametrically specified trend. By using penalized splines as the representation for the non-parametric trend, it is possible to express the non-parametric small area estimation problem as a mixed effect model regression. The resulting model is readily fitted by using existing model fitting approaches such as restricted maximum likelihood. We present…

高维小样本分类器相对表现的理论度量AI 解读

Summary We suggest a technique, related to the concept of ‘detection boundary’ that was developed by Ingster and by Donoho and Jin, for comparing the theoretical performance of classifiers constructed from small training samples of very large vectors. The resulting ‘classification boundaries’ are obtained for a variety of distance-based methods, including the support vector machine, distance-weighted discrimination and kth-nearest-neighbour classifiers, for thresholded forms…

Logistic 回归中的组 LassoAI 解读

Summary The group lasso is an extension of the lasso to do variable selection on (predefined) groups of variables in linear regression models. The estimates have the attractive property of being invariant under groupwise orthogonal reparameterizations. We extend the group lasso to logistic regression models and present an efficient algorithm, that is especially suitable for high dimensional problems, which can also be applied to generalized linear models to solve the…

Rodeo：稀疏、贪心的非参数回归AI 解读

We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method—called rodeo (regularization of derivative expectation operator)—conducts a sequence of hypothesis tests to threshold derivatives,…

强依赖下 higher criticism 检验的性质AI 解读

The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of…

混合收敛速率的渐近分析AI 解读

A general method is presented for deriving the limiting behavior of estimators that are defined as the values of parameters optimizing an empirical criterion function. The asymptotic behavior of such estimators is typically deduced from uniform limit theorems for rescaled and reparametrized criterion functions. The new method can handle cases where the standard approach does not yield the complete limiting behavior of the estimator. The asymptotic analysis depends on a…

大型协方差矩阵的正则化估计AI 解读

This paper considers estimating a covariance matrix of p variables from n observations by either banding or tapering the sample covariance matrix, or estimating a banded version of the inverse of the covariance. We show that these estimates are consistent in the operator norm as long as (log p)/n→0, and obtain explicit rates. The results are uniform over some fairly natural well-conditioned families of covariance matrices. We also introduce an analogue of the Gaussian white…

半参数回归建模中的变量选择AI 解读

In this paper, we are concerned with how to select significant variables in semiparametric modeling. Variable selection for semiparametric regression models consists of two components: model selection for nonparametric components and select significant variables for parametric portion. Thus, it is much more challenging than that for parametric models such as linear models and generalized linear models because traditional variable selection procedures including stepwise…

惩罚比例风险模型中基于路径的变量选择AI 解读

We propose an efficient and adaptive shrinkage method for variable selection in the Cox model. The method constructs a piecewise-linear regularization path connecting the maximum partial likelihood estimator and the origin. Then a model is selected along the path. We show that the constructed path is adaptive in the sense that, with a proper choice of regularization parameter, the fitted model works as well as if the true underlying submodel were given in advance. A modified…

利用平滑间隔函数分析提升算法AI 解读

We introduce a useful tool for analyzing boosting algorithms called the “smooth margin function,” a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, “coordinate ascent boosting” and “approximate coordinate ascent boosting,” which are similar to Freund and Schapire’s AdaBoost algorithm and Breiman’s arc-gv algorithm. We give convergence rates to the maximum margin solution for both of our…

非线性加性自回归模型的样条回拟合核平滑AI 解读

Application of nonparametric and semiparametric regression techniques to high-dimensional time series data has been hampered due to the lack of effective tools to address the “curse of dimensionality.” Under rather weak conditions, we propose spline-backfitted kernel estimators of the component functions for the nonlinear additive time series data that are both computationally expedient so they are usable for analyzing very high-dimensional time series, and theoretically…