Noncrossing quantile regression with an emphasis on model misspecification is investigated. While many sophisticated methods for noncrossing quantile have been developed under the assumption of correct model specification, model misspecification and extrapolation are two issues rarely considered in the literature. In this paper, a monotonicity representation for quantile regression models is obtained under simplex embedding, which leads to the simplex quantile regression…
The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet the degree of sparsity itself is…
Model averaging has long been proposed as a powerful alternative to model selection in regression analysis. However, how well it performs in high-dimensional regression is still poorly understood. Recently, Ando and Li [J. Amer. Statist. Assoc. 109 (2014) 254–265] introduced a new method of model averaging that allows the number of predictors to increase as the sample size increases. One notable feature of Ando and Li’s method is the relaxation on the total model weights so…
This article considers high-dimensional regression problems in which the number of predictors p exceeds the sample size n. We develop a model-averaging procedure for high-dimensional regression problems. Unlike most variable selection studies featuring the identification of true predictors, our focus here is on the prediction accuracy for the true conditional mean of y given the p predictors. Our method consists of two steps. The first step is to construct a class of…
Bayesian models have been developed that find ungapped motifs in multiple protein sequences. In this article, we extend the model to allow for deletions and insertions in motifs. Direct generalization of the ungapped algorithm, based on Gibbs sampling, proved unsuccessful because the configuration space became much larger. To alleviate the convergence difficulty, a two-stage procedure is introduced. At the first stage, we develop a method called entropy filtering, which…
In this article we present a framework for parsimonious modeling of large ensembles of time series. The idea is to identify a small number of stochastic time series components such that each series in the ensemble is a weighted sum of series-specific realizations of the components. We present an estimation procedure that is computationally tractable for large datasets. Through simulations, we argue that the estimators perform well. We illustrate the method using two…
Digital deconvolution concerns the restoration of an underlying discrete signal from a blurred noisy observation sequence. The problem can be formulated in a Bayesian framework. As is usual in the Bayesian context, the computation of relevant posterior quantities is the major challenge. Previous work made substantial progress toward making this computation feasible, most notably through the Gibbs sampling approach of Chen and Li and the sequential importance sampling…
An interactive approach to tree-structured regression is introduced. Unlike other procedures driven by cost optimization, this approach focuses on the exploration of geometric information in the data. The procedure begins with finding a direction along which the regression surface bends the most. This direction is used for splitting the data into two regions. Within each region, another direction is found, and the partition is made in the same manner. The process continues…
Chun-Houh Chen · Ker-Chau Li · Jane-Ling Wang·刊期 1999-03-01 · 卷 27（1）·The Annals of Statistics已编入正刊
删失回归数据的降维AI 解读
Without parametric assumptions, high-dimensional regression analysis is already complex. This is made even harder when data are subject to censoring. In this article, we seek ways of reducing the dimensionality of the regressor before applying nonparametric smoothing techniques. If the censoring time is independent of the lifetime, then the method of sliced inverse regression can be applied directly. Otherwise, modification is needed to adjust for the censoring bias. A key…
It is not uncommon to find nonlinear patterns in the scatterplots of regressor variables. But how such findings affect standard regression analysis remains largely unexplored. This article offers a theory on nonlinear confounding, a term for describing the situation where a certain nonlinear relationship in regressors leads to difficulties in modeling and related analysis of the data. The theory begins with a measure of nonlinearity between two regressor variables. It is…
This paper studies the shapes of low dimensional projections from high dimensional data. After standardization, let $\mathbf{x}$ be a $p$-dimensional random variable with mean zero and identity covariance. For a projection $\beta'\mathbf{x}, \|\beta\| = 1$, find another direction $b$ so that the regression curve of $b'\mathbf{x}$ against $\beta'\mathbf{x}$ is as nonlinear as possible. We show that when the dimension of $\mathbf{x}$ is large, for most directions $\beta$ even…
In comparing two treatment regimens, a matched-pair design is often used for nonhomogeneous experimental units. A standard method of analyzing the resulting data is to consider the difference within pairs, so that the nuisance parameters due to the heterogeneity between pairs can be eliminated and the parameter of interest—the treatment effect—can be estimated. This procedure becomes invalid when the data are subject to censoring, however. In addition, the available…
A general nonlinear regression problem is considered with measurement error in the predictors. We assume that the response is related to an unknown linear combination of a multidimensional predictor through an unknown link function. Instead of observing the predictor, we instead observe a surrogate with the property that its expectation is linearly related to the true predictor with constant variance. We identify an important transformation of the surrogate variable. Using…
Modern graphical tools have enhanced our ability to learn many things from data directly. With much user-friendly graphical software available, we are encouraged to plot a lot more often than before. The benefits from direct interaction with graphics have been enormous. But trailing behind these high-tech advances is the issue of appropriate guidance on what to plot. There are too many directions to project a high-dimensional data set and unguided plotting can be…
Consider a general regression model of the form $y = g(\alpha + \mathbf{x}'\beta, \varepsilon)$, with an arbitrary and unknown link function $g$. We study a link-free method, the slicing regression, for estimating the direction of $\beta$. The method is easy to implement and does not require iterative computation. First, we estimate the inverse regression function $E(\mathbf{x}\mid y)$ using a step function. We then estimate $\Gamma = \operatorname{Cov}\lbrack E(\mathbf{x}\mid y)\rbrack$…
We study the behavior of regression analysis when there might be some violation of the assumed link function, the functional form of the model which relates the outcome variable $y$ to the regressor variable $\mathbf{x}$ and the random error. We allow the true link function to be completely arbitrary, except that $y$ depends on $\mathbf{x}$ only through a linear combination $\beta\mathbf{x}$. The slope vector $\beta$ is identified only up to a multiplicative scalar. Under…
The problem of constructing honest confidence regions for nonparametric regression is considered. A lower rate of convergence, $n^{-1/4}$, for the size of the confidence region is established. The achievability of this rate is demonstrated using Stein's estimates and the associated unbiased risk estimates. Practical implications are discussed.
$C_p, C_L$, cross-validation and generalized cross-validation are useful data-driven techniques for selecting a good estimate from a proposed class of linear estimates. The asymptotic behaviors of these procedures are studied. Some easily interpretable conditions are derived to demonstrate the asymptotic optimality. It is argued that cross-validation and generalized cross-validation can be viewed as some special ways of applying $C_L$. Applications in nearest-neighbor…
This paper attempts to fill the gap between some different and conflicting approaches in survey sampling. Based on a fixed population regression-type model, a class of optimality criteria similar to those well accepted in optimal experimental design theory is introduced. The minimax and superpopulation approaches in survey sampling turn out to correspond to two extreme criteria in the proposed class. This helps understand the role of randomization. The strategy of simple…
SUMMARY This paper attempts to fill the gap between some different and conflicting approaches in survey sampling. Based on a fixed population regression-type model, a class of optimality criteria similar to those well accepted in optimal experimental design theory is introduced. The minimax and superpopulation approaches in survey sampling turn out to correspond to two extreme criteria in the proposed class. This helps understand the role of randomization. The strategy of…
The asymptotic optimality of Mallows' $C_L$ and generalized cross-validation is demonstrated in the setting of ridge regression. An application is made to spline smoothing in nonparametric regression. A counterexample is given to help understand why sometimes GCV may not be asymptotically optimal. The coefficient of variation for the eigenvalues of the information matrix must be large in order to guarantee the optimality of GCV. The proff is based on the connection between…
This paper studies the shapes of low dimensional projections from high dimensional data. After standardization, let x be a p-dimensional random variable with mean zero and identity covariance. For a projection β′x,∥β∥=1, find another direction b so that the regression curve of b′x against β′x is as nonlinear as possible. We show that when the dimension of x is large, for most directions β even…
Consider a general regression model of the form y=g(α+x′β,ε), with an arbitrary and unknown link function g. We study a link-free method, the slicing regression, for estimating the direction of β. The method is easy to implement and does not require iterative computation. First, we estimate the inverse regression function E(x∣y) using a step function. We then estimate Γ=Cov[E(x∣y)]…
We study the behavior of regression analysis when there might be some violation of the assumed link function, the functional form of the model which relates the outcome variable y to the regressor variable x and the random error. We allow the true link function to be completely arbitrary, except that y depends on x only through a linear combination βx. The slope vector β is identified only up to a multiplicative scalar. Under…
The problem of constructing honest confidence regions for nonparametric regression is considered. A lower rate of convergence, n−1/4, for the size of the confidence region is established. The achievability of this rate is demonstrated using Stein's estimates and the associated unbiased risk estimates. Practical implications are discussed.
Asymptotic Optimality for Cp,CL, Cross-Validation and Generalized Cross-Validation: Discrete Index Set
Cp,CL, cross-validation and generalized cross-validation are useful data-driven techniques for selecting a good estimate from a proposed class of linear estimates. The asymptotic behaviors of these procedures are studied. Some easily interpretable conditions are derived to demonstrate the asymptotic optimality. It is argued that cross-validation and generalized cross-validation can be viewed as some special ways of applying CL. Applications in nearest-neighbor…
Asymptotic Optimality of CL and Generalized Cross-Validation in Ridge Regression with Application to Spline Smoothing
The asymptotic optimality of Mallows' CL and generalized cross-validation is demonstrated in the setting of ridge regression. An application is made to spline smoothing in nonparametric regression. A counterexample is given to help understand why sometimes GCV may not be asymptotically optimal. The coefficient of variation for the eigenvalues of the information matrix must be large in order to guarantee the optimality of GCV. The proff is based on the connection between…