An asymptotic expansion is provided for the transformation kernel density estimator introduced by Ruppert and Cline. Let $h_k$ be the band-width used in the $k$th iteration, $k = 1,2,\ldots, t$. If all bandwidths are of the same order, the leading bias term of the $l$th derivative of the $t$th iterate of the density estimator has the form $\bar{b}^{(l)}_t(x) \pi^t_{k=1} h^2_k$, where the bias factor $\bar{b}_t(x)$ depends on the second moment of the kernel $K$, as well as on…
The transform-both-sides (TBS) regression model developed by Carroll and Ruppert is applicable when the relationship between the median response and the independent variables has been identified. Several different families of transformations, such as the Box-Cox power family, have been considered in the parametric approach to this model. In this article, we propose a nonparametric estimator of the transformation in the TBS model allowing general smooth monotonic…
The transform-both-sides (TBS) regression model developed by Carroll and Ruppert is applicable when the relationship between the median response and the independent variables has been identified. Several different families of transformations, such as the Box-Cox power family, have been considered in the parametric approach to this model. In this article, we propose a nonparametric estimator of the transformation in the TBS model allowing general smooth monotonic…
A modification of kernel density estimation is proposed. The first step is ordinary kernel estimation of the density and its cdf. In the second step the data are transformed, using this estimated cdf, to an approximate uniform (or normal or other target) distribution. The density and cdf of the transformed data are then estimated by the kernel method and, by change of variable, converted to new estimates of the density and the cdf of the original data. This process is…
The breakdown point is considered an important measure of the robustness of a linear regression estimator. This article addresses the concept of breakdown in nonlinear regression. Because it is not invariant to nonlinear reparameterization, the usual definition of the breakdown point in linear regression is inadequate for nonlinear regression. The original definition of breakdown due to Hampel is more suitable for nonlinear problems but may indicate breakdown when the fitted…
We investigate the design of experiments for the nonparametric estimation of the root of an unknown regression function. Approaches to this problem include the Robbins-Monro (1951), Venter (1967), and Lai-Robbins (1979, 1981) stochastic approximation procedures and Wu's (1985, 1986) sequential maximum likelihood estimators. Because the regression function is not assumed to belong to a parametric family, only experimentation near to the root is informative and the sequential…
We investigate the design of experiments for the nonparametric estimation of the root of an unknown regression function. Approaches to this problem include the Robbins-Monro (1951), Venter (1967), and Lai-Robbins (1979, 1981) stochastic approximation procedures and Wu's (1985, 1986) sequential maximum likelihood estimators. Because the regression function is not assumed to belong to a parametric family, only experimentation near to the root is informative and the sequential…
Introduction. Generalized Least Squares and the Analysis of Heteroscedasticity. Estimation and Inference for Variance Functions. The Transform-Both-Sides Methodology. Combining Transformations and Weighting. Influence and Robustness. Technical Complements. Some Open Problems. References. Index.
This article considers transformations in regression to eliminate skewness and heteroscedasticity of the response. We work with the transform-both-sides model where the relationship between the median response and the independent variables has been identified, at least tentatively. To preserve this relationship, the response and the regression model are transformed in the same way. Extending the work of others for the location parameter case, we propose an estimator that…
In weighted least squares, it is typical that the weights are unknown and must be estimated. Most packages provide standard errors assuming that the weights are known. This is fine for sufficiently large sample sizes, but what about for small-to-moderate sample sizes? The investigation of this article into the effect of estimating weights proceeds under the assumption typical in practice—that one has a parametric model for the variance function. In this context, generalized…
Douglas G. Simpson · Raymond J. Carroll · David Ruppert·刊期 1987-06-01 · 卷 15(2)·The Annals of Statistics已编入正刊
离散数据的 M 估计:渐近分布理论及其含义AI 解读
The asymptotic distribution of an $M$-estimator is studied when the underlying distribution is discrete. Asymptotic normality is shown to hold quite generally within the assumed parametric family. When the specification of the model is inexact, however, it is demonstrated that an $M$-estimator whose score function is not everywhere differentiable, e.g., a Huber estimator, has a nonnormal limiting distribution at certain distributions, resulting in unstable inference in the…
LEONARD A. STEFANSKI · RAYMOND J. CARROLL · DAVID RUPPERT·刊期 1986 · 卷 73(2)·Biometrika已编入正刊
广义线性模型的最优有界得分函数及其在逻辑回归中的应用AI 解读
We study optimally bounded score functions for estimating regression parameters in a generalized linear model. Our work extends results obtained by Krasker & Welsch (1982) for the linear model and provides a simple proof of Krasker & Welsch's first-order condition for strong optimality. The application of these results to logistic regression is studied in some detail with an example given comparing the bounded-influence estimator with maximum likelihood.
Under an age replacement policy, a stochastically failing unit is replaced at failure or after being in service for $t$ units of time, whichever comes first. An important problem is the estimation of $\phi^\ast,$ the optimal replacement time when the form of the failure distribution is unknown. Here, $\phi^\ast$ is optimal in the sense that it is the replacement time that achieves the smallest long-run expected cost. It is shown that substantial cost savings can be effected…
Recently, Krasker and Welsch (1982) considered a class of bounded-influence regression estimators. They showed that within this class the so-called Krasker—Welsch estimator is the only solution to a first-order necessary condition for strong optimality, that is, for minimizing, in the sense of positive definiteness, the asymptotic covariance matrix. However, whether any strongly optimal estimator in fact exists remained an open question. In this article, an example is given…
Recently, Krasker and Welsch (1982) considered a class of bounded-influence regression estimators. They showed that within this class the so-called Krasker—Welsch estimator is the only solution to a first-order necessary condition for strong optimality, that is, for minimizing, in the sense of positive definiteness, the asymptotic covariance matrix. However, whether any strongly optimal estimator in fact exists remained an open question. In this article, an example is given…
We investigate power transformations in nonlinear regression problems when there is a physical model for the response but little understanding of the underlying error structure. In such circumstances, and unlike the ordinary power transformation model, both the response and the model must be transformed simultaneously and in the same way. We show by an asymptotic theory and a small Monte Carlo study that for estimating the model parameters there is little cost for not…
We consider a linear model with normally distributed but heteroscedastic errors. When the error variances are functionally related to the regression parameter, one can use either maximum likelihood or generalized least squares to estimate the regression parameter. We show that likelihood is more sensitive to small misspecifications in the functional relationship between the error variances and the regression parameter.
We consider a heteroscedastic linear model in which the variances are given by a parametric function of the mean responses and a parameter $\theta$. We propose robust estimates for the regression parameter $\beta$ and show that, as long as a reasonable starting estimate of $\theta$ is available, our estimates of $\beta$ are asymptotically equivalent to the natural estimate obtained with known variances. A particular method for estimating $\theta$ is proposed and shown by…
Let $S$ be a set, $R$ the real line, and $M$ a real function on $R \times S$. Assume there exists a real function, $f$, on $S$ such that $(x - f(s))M(x, s) \geq 0$ for all $x$ and $s$. Initially neither $M$ nor $f$ are known. The goal is to estimate $f$. At time $n, s_n$ (a value in $S$) is observed, $x_n$ (a real number) is chosen, and an unbiased estimator of $M(x_n, s_n)$ is observed. This problem has applications, for example, to process control. In a previous paper the…
We extend Bickel's tests for heteroscedasticity to include wider classes of test statistics and fitting methods. The test statistics include those based on Huber's function, while the fitting techniques include Huber's Proposal 2 for robust regression.
We consider two methods of defining a regression analog to a trimmed mean. The first was suggested by Koenker and Bassett and uses their concept of regression quantiles. Its asymptotic behavior is completely analogous to that of a trimmed mean. The second method uses residuals from a preliminary estimator. Its asymptotic behavior depends heavily on the preliminary estimate; it behaves, in general, quite differently than the estimator proposed by Koenker and Bassett, and it…
This paper considers Robbins-Monro stochastic approximation when the regression function changes with time. At time $n$, one can select $X_n$ and observe an unbiased estimator of the regression function evaluated at $X_n$. Let $\theta_n$ be the root of the regression function at time $n$. Our goal is to select the sequence $X_n$ so that $X_n - \theta_n$ converges to 0. It is assumed that $\theta_n = f(s_n)$ for $s_n$ known at time $n$ and $f$ an unknown element of a class of…
An asymptotic expansion is provided for the transformation kernel density estimator introduced by Ruppert and Cline. Let hk be the band-width used in the kth iteration, k=1,2,…,t. If all bandwidths are of the same order, the leading bias term of the lth derivative of the tth iterate of the density estimator has the form bˉt(l)(x)πk=1thk2, where the bias factor bˉt(x) depends on the second moment of the kernel K, as well as on…
M-Estimation for Discrete Data: Asymptotic Distribution Theory and Implications
The asymptotic distribution of an M-estimator is studied when the underlying distribution is discrete. Asymptotic normality is shown to hold quite generally within the assumed parametric family. When the specification of the model is inexact, however, it is demonstrated that an M-estimator whose score function is not everywhere differentiable, e.g., a Huber estimator, has a nonnormal limiting distribution at certain distributions, resulting in unstable inference in the…
Under an age replacement policy, a stochastically failing unit is replaced at failure or after being in service for t units of time, whichever comes first. An important problem is the estimation of ϕ∗, the optimal replacement time when the form of the failure distribution is unknown. Here, ϕ∗ is optimal in the sense that it is the replacement time that achieves the smallest long-run expected cost. It is shown that substantial cost savings can be effected…
We consider a heteroscedastic linear model in which the variances are given by a parametric function of the mean responses and a parameter θ. We propose robust estimates for the regression parameter β and show that, as long as a reasonable starting estimate of θ is available, our estimates of β are asymptotically equivalent to the natural estimate obtained with known variances. A particular method for estimating θ is proposed and shown by…
Let S be a set, R the real line, and M a real function on R×S. Assume there exists a real function, f, on S such that (x−f(s))M(x,s)≥0 for all x and s. Initially neither M nor f are known. The goal is to estimate f. At time n,sn (a value in S) is observed, xn (a real number) is chosen, and an unbiased estimator of M(xn,sn) is observed. This problem has applications, for example, to process control. In a previous paper the…
This paper considers Robbins-Monro stochastic approximation when the regression function changes with time. At time n, one can select Xn and observe an unbiased estimator of the regression function evaluated at Xn. Let θn be the root of the regression function at time n. Our goal is to select the sequence Xn so that Xn−θn converges to 0. It is assumed that θn=f(sn) for sn known at time n and f an unknown element of a class of…