In an 1886 paper, Edgeworth developed a method for simulating time series processes with substantial dependence. A version of this process with normal errors has the same means and covariance structure as an AR(1) process, but is actually a mixture of a very large number of processes, some of which are not stationary. That is, joint distributions of lag 3 or greater are not normal but are mixtures of normals (even though all successive pairs are bivariate normal). Thus, it…
Traditionally, assessing the accuracy of inference based on regression quantiles has relied on the Bahadur representation. This provides an error of order $n^{-1/4}$ in normal approximations, and suggests that inference based on regression quantiles may not be as reliable as that based on other (smoother) approaches, whose errors are generally of order $n^{-1/2}$ (or better in special symmetric cases). Fortunately, extensive simulations and empirical applications show that…
Using quantile regression to analyze survival times offers an valuable complement to traditional Cox proportional hazards modelling. Unfortunately, this approach has been hampered by the lack of a conditional quantile estimator for censored data that is directly analogous to the Kaplan–Meier estimator and applies under standard assumptions for censored regression models. Here a recursively reweighted estimator of the regression quantile process is developed as a direct…
We suggest interpolation methods for overcoming the problem of sparse design in local linear smoothing. These methods are based on simple rules, determined by the kernel and bandwidth, for deciding when and where pseudo-design points should be added to augment the original design sequence. New ordinates for the added design points are computed by simple interpolation, then local linear smoothing is applied directly to the expanded dataset. The method is competitive with…
Quantile smoothing splines were introduced by Koenker, Ng and Portnoy as natural and appealing estimates of conditional quantiles of response variables. The natural setting for the problem considers minimization of a weighted combination of a "fit" penalty and a "roughness" penalty over the space of functions whose derivatives have bounded variation. Although this space is not traditional, Shen has shown recently that the quantile smoothing splines do indeed converge at the…
The problem of combining high efficiency with high breakdown properties for regression estimators has piqued the interest of statisticians for some time. One proposal specifically suggested by Rousseeuw and Leroy is to use the least median of squares estimator, omit observations whose residuals are larger than some constant cut-off value and apply least squares to the remaining observations. Although this proposal does retain high breakdown point, it actually converges no…
Robust alternatives to the seemingly unrelated regression (SUR) estimator of Zellner (1962) are proposed for the classical multivariate regression model. These weighted M estimators achieve an asymptotic covariance matrix analogous to that of the SUR estimator. Comparisons for the l 1, least absolute deviation, case are made with the efficient estimator in the case of elliptically contoured distributions. An example reanalyzing the Grunfeld investment data using a smooth “l…
Robust alternatives to the seemingly unrelated regression (SUR) estimator of Zellner (1962) are proposed for the classical multivariate regression model.These weighted M-estimators achieve an asymptotic covariance matrix analogous to that of the SUR estimator.Comparisons for the l lt least absolute deviation, case are made with the efficient estimator in the case of elliptically contoured distributions.An example reanalyzing the Grunfeld investment data using a smooth…
Xuming He · Jana Jureckova · Roger Koenker · Stephen Portnoy·刊期 1990-09 · 卷 58（5）·Econometrica已编入正刊
回归估计量的尾部行为及其崩溃点AI 解读
A measure of finite sample estimator performance based on the probability of large deviations is introduced. The tail performance of the least squares estimator is studied in detail, and the authors find that it achieves good tail performance under strictly Gaussian conditions, but performance is extremely poor in the case of heavy-tailed error distributions. Turning to the tail behavior of various robust estimators, they focus on tail performance under heavy (algebraic)…
Asymptotically efficient (adaptive) estimators for the slope parameters of the linear regression model are constructed based upon the "regression quantile" statistics suggested by Koenker and Bassett. The estimators are natural analogues of the adaptive $L$-estimators of location of Sacks, but employ kernel-density type estimators of the optimal $L$-estimator weight function.
Consider a sample of size $n$ from a regular exponential family in $p_n$ dimensions. Let $\hat\theta_n$ denote the maximum likelihood estimator, and consider the case where $p_n$ tends to infinity with $n$ and where $\{\theta_n\}$ is a sequence of parameter values in $R^{p_n}$. Moment conditions are provided under which $\|\hat\theta_n - \theta_n\| = O_p(\sqrt{p_n/n})$ and $\|\hat\theta_n - \theta_n - \overline{X}_n\| = O_p (p_n/n)$, where $\overline{X}_n$ is the sample…
Linear combinations of order statistics, or L-estimators, have played an extremely important role in the development of robust methods for the one-sample problem. We suggest analogs of L-estimators for the parameters of the linear model based on the p-dimensional “regression quantiles” proposed by Koenker and Bassett (1978). A uniform, Bahadur-type asymptotic representation of regression quantiles is established, and this yields a general asymptotic theory of L-estimators…
Consider a regression model $Y_i=x'_i\beta+R_i$, i=1,\cdots,n, where ${R_i}$ are i.i.d. with c.d.f., F; $x_i\inR^p$ and $\beta\inR^p$. Let $\hat{\beta}$ be a M-estimator defined using kernel, $\psi$; let $\hat{F}_n(x)$ denote the empiric distribution of the residuals, $Y_i-x'_i\hat{\beta}$ and let $\hat{F}^\ast_n$ be the empiric c.d.f. of the errors, ${R_i}$. Under suitable smoothness conditions on $\psi$, F, and the density F' = f and conditions requiring essentially that ${x_i}$…
In a general linear model, $Y = X\beta + R$ with $Y$ and $R n$-dimensional, $X$ a $n \times p$ matrix, and $\beta p$-dimensional, let $\hat\beta$ be an $M$ estimator of $\beta$ satisfying $0 = \sum x_i\psi(y_i - x'_i\beta)$. Let $p \rightarrow \infty$ such that $(p \log n)^{3/2} /n \rightarrow 0$. Then $\max_i|x'_i(\hat{\beta} - \beta)| \rightarrow _P 0$, and it is possible to find a uniform normal approximation for the distribution of $\hat{\beta}$ under which arbitrary…
Consider the general linear model $Y = x\beta + R$ with $Y$ and $R n$-dimensional, $\beta p$-dimensional, and $X$ an $n \times p$ matrix with rows $x'_i$. Let $\psi$ be given and let $\hat\beta$ be an $M$-estimator of $\beta$ satisfying $0 = \sum x_i\psi(Y_i - x'_i\hat\beta)$. Previous authors have considered consistency and asymptotic normality of $\hat\beta$ when $p$ is permitted to grow, but they have required at least $p^2/n \rightarrow 0$. Here the following result is…
Consider a regular $p$-dimensional exponential family such that either the distributions are concentrated on a lattice or they have a component whose $k$-fold convolution has a bounded density with respect to Lebesgue measure. Then, if a parametric function has an unbiased estimator, the minimum variance unbiased estimators are asymptotically equivalent to the maximum likelihood estimators; and, hence, are asymptotically efficient. Examples are given to show that a condition…
Nearly root-n approximation for regression quantile processes
Traditionally, assessing the accuracy of inference based on regression quantiles has relied on the Bahadur representation. This provides an error of order n−1/4 in normal approximations, and suggests that inference based on regression quantiles may not be as reliable as that based on other (smoother) approaches, whose errors are generally of order n−1/2 (or better in special symmetric cases). Fortunately, extensive simulations and empirical applications show that…
Asymptotically efficient (adaptive) estimators for the slope parameters of the linear regression model are constructed based upon the "regression quantile" statistics suggested by Koenker and Bassett. The estimators are natural analogues of the adaptive L-estimators of location of Sacks, but employ kernel-density type estimators of the optimal L-estimator weight function.
Consider a sample of size n from a regular exponential family in pn dimensions. Let θ^n denote the maximum likelihood estimator, and consider the case where pn tends to infinity with n and where {θn} is a sequence of parameter values in Rpn. Moment conditions are provided under which ∥θ^n−θn∥=Op(pn/n) and ∥θ^n−θn−Xn∥=Op(pn/n), where Xn is the sample…
Consider a regression model Yi=xi′β+Ri, i=1,\cdots,n, where Ri are i.i.d. with c.d.f., F; $x_i\inR^p$ and $\beta\inR^p$. Let β^ be a M-estimator defined using kernel, ψ; let F^n(x) denote the empiric distribution of the residuals, Yi−xi′β^ and let F^n∗ be the empiric c.d.f. of the errors, Ri. Under suitable smoothness conditions on ψ, F, and the density F' = f and conditions requiring essentially that xi…
Asymptotic Behavior of M Estimators of p Regression Parameters when p2/n is Large; II. Normal Approximation
In a general linear model, Y=Xβ+R with Y and Rn-dimensional, X a n×p matrix, and βp-dimensional, let β^ be an M estimator of β satisfying 0=∑xiψ(yi−xi′β). Let p→∞ such that (plogn)3/2/n→0. Then maxi∣xi′(β^−β)∣→P0, and it is possible to find a uniform normal approximation for the distribution of β^ under which arbitrary…
Asymptotic Behavior of M-Estimators of p Regression Parameters when p2/n is Large. I. Consistency
Consider the general linear model Y=xβ+R with Y and Rn-dimensional, βp-dimensional, and X an n×p matrix with rows xi′. Let ψ be given and let β^ be an M-estimator of β satisfying 0=∑xiψ(Yi−xi′β^). Previous authors have considered consistency and asymptotic normality of β^ when p is permitted to grow, but they have required at least p2/n→0. Here the following result is…
Consider a regular p-dimensional exponential family such that either the distributions are concentrated on a lattice or they have a component whose k-fold convolution has a bounded density with respect to Lebesgue measure. Then, if a parametric function has an unbiased estimator, the minimum variance unbiased estimators are asymptotically equivalent to the maximum likelihood estimators; and, hence, are asymptotically efficient. Examples are given to show that a condition…