In the setting of kernel density estimation, data-driven bandwidth, i.e., smoothing parameter, selectors are considered. It is seen that there is a well-defined, and surprisingly restrictive, bound on the rate of convergence of any automatic bandwidth selection method to the optimum. The method of least squares cross-validation achieves this bound.
In the setting of nonparametric multivariate density estimation, theorems are established which allow a comparison of the Kullback-Leibler and the least-squares cross-validation methods of smoothing parameter selection. The family of delta sequence estimators (including kernel, orthogonal series, histogram and histospline estimators) is considered. These theorems also show that either type of cross validation can be used to compare different estimators (e.g., kernel versus…
We consider the estimation problem for the parameter $\vartheta_0$ of a stationary ARMA $(p, q)$ process, with independent and identically, but not necessary normally distributed errors. First we prove local asymptotic normality (LAN) for this model. Then we construct locally asymptotically minimax (LAM) estimators, which asymptotically achieve the smallest possible covariance matrix. Utilizing these, we finally obtain strongly adaptive estimators, by using usual kernel…
A new sieve estimator for the mean function $m(t)$ of a general Gaussian process of known covariance is presented. The estimator $\hat{m}(t)$ is given explicitly from the data and has a simple distribution. It is shown that $\hat{m}(t)$ is asymptotically unbiased and consistent (weakly and in mean square) at each $t$, and that $\hat{m}$ is strongly consistent for $m$ in an appropriate norm. No assumptions are made about the "time" parameter or the covariance.
Weighted local regression, a popular technique for smoothing scatterplots, is shown to be asymptotically equivalent to certain kernel smoothers. Since both methods are local weighted averages of the data, it is proved that in the fixed design regression model, given a weighted local regression procedure with any weight function, there is a corresponding kernel method such that the quotients of weights distributed by both methods tend uniformly to 1 as the number of…
Differentiation of Integrals Consistency Lower Bounds for Rates of Convergence Rates of Convergence in L1 The Automatic Kernel Estimate: L1 and Pointwise Convergence Estimates Related to the Kernel Estimate and the Histogram Estimate Simulation, Inequalities, and Random Variate Generation The Transformed Kernel Estimate Applications in Discrimination Operations on Density Estimates Estimators Based on Orthogonal Series Index.
Craig F. Ansley · Robert Kohn·刊期 1987-03 · 卷 74(1)·Biometrika已编入正刊
状态空间模型的高效广义交叉验证AI 解读
For a signal plus noise model with a state space representation, an efficient procedure is given for obtaining the trace of the influence matrix, where the influence matrix expresses the estimated signal vector as a linear combination of the observed data. This allows an O(n) evaluation of the generalized cross-validation criterion function. Our approach is very efficient, requiring the addition of only one equation to the ordinary Kalman filter, and extends to models with…
SUMMARY A nonparametric estimate is proposed for the hazard function using penalized likelihood methods. The estimate turns out to be a hyperbolic spline function which is continuous with its first derivative discontinuous only at a finite number of points. It always assumes positive values, unlike Anderson and Senthilselvan’s (1980) quadratic spline estimate. This method is modified for the estimation of the intensity function of a nonstationary Poisson process. An example…
HANS-GEORG MÜLLER · U. STADTMÜLLER · THOMAS SCHMITT·刊期 1987 · 卷 74(4)·Biometrika已编入正刊
含噪数据导数估计的带宽选择与置信区间AI 解读
We propose a method for automatic bandwidth selection for kernel estimators of derivatives of a regression function. The finite sample behaviour of this new method is compared with that of other methods in a Monte Carlo Study. The automatic estimation of derivatives can be employed for the construction of asymptotic local confidence intervals for the nonparametric estimate of the regression function and its first derivatives.
PETER HALL · G. S. WATSON · JAVIER CABRERA·刊期 1987 · 卷 74(4)·Biometrika已编入正刊
球面数据的核密度估计AI 解读
We study two natural classes of kernel density estimators for use with spherical data. Members of both classes have already been used in practice. The classes have an element in common, but for the most part they are disjoint. However, all members of the first class are asymptotically equivalent to one another, and to a single element of the second class. In this sense the second class ‘contains’ the first. It includes some estimators which out-perform all those in the first…
CRAIG F. ANSLEY · ROBERT KOHN·刊期 1987 · 卷 74(1)·Biometrika已编入正刊
状态空间模型的高效广义交叉验证AI 解读
For a signal plus noise model with a state space representation, an efficient procedure is given for obtaining the trace of the influence matrix, where the influence matrix expresses the estimated signal vector as a linear combination of the observed data. This allows an O(n) evaluation of the generalized cross-validation criterion function. Our approach is very efficient, requiring the addition of only one equation to the ordinary Kalman filter, and extends to models with…
In order to improve the rate of decrease of the IMSE for nonparametric kernel density estimators with nonrandom bandwidth beyond $O(n^{-4/5})$ all current methods must relax the constraint that the density estimate be a bona fide function, that is, be nonnegative and integrate to one. In this paper we show how to achieve similar improvement without relaxing any of these constraints. The method can also be applied for orthogonal series, adaptive orthogonal series, spline,…
An Edgeworth expansion with remainder $o(N^{-1})$ is established for a $U$-statistic with a kernel $h$ of degree 2. The assumptions involved appear to be very mild; in particular, the common distribution of the summands $h(X_i, X_j)$ is not assumed to be smooth.
Outliers in sample data are a perennial problem for applied survey statisticians. Moreover, it is a problem for which traditional sample survey theory offers no real solution, beyond the sensible advice that such sample elements should not be weighted to their fullest extent in estimation. Sample outliers can be identified as of two basic types. Here we are concerned with the first type, which may conveniently be termed representative outliers. These are sample elements with…
A nonparametric estimator of residual variance in nonlinear regression is proposed. It is based on local linear fitting. Asymptotically the estimator has a small bias, but a larger variance compared with the parametric estimator in linear regression. Finite sample properties are investigated in a simulation study, including a comparison with other nonparametric estimators. The method is also useful for spotting heteroscedasticity and outliers in the residuals at an early…
In this paper, we study the local deviations of the empirical measure defined by the Kaplan-Meier (1958) estimator for the survival function. The results are applied to derive best rates of convergence for kernel estimators for the density and hazard rate function in the random censorship model.
Consider a regression model $Y_i=x'_i\beta+R_i$, i=1,\cdots,n, where ${R_i}$ are i.i.d. with c.d.f., F; $x_i\inR^p$ and $\beta\inR^p$. Let $\hat{\beta}$ be a M-estimator defined using kernel, $\psi$; let $\hat{F}_n(x)$ denote the empiric distribution of the residuals, $Y_i-x'_i\hat{\beta}$ and let $\hat{F}^\ast_n$ be the empiric c.d.f. of the errors, ${R_i}$. Under suitable smoothness conditions on $\psi$, F, and the density F' = f and conditions requiring essentially that ${x_i}$…
The asymptotic optimality of Mallows' $C_L$ and generalized cross-validation is demonstrated in the setting of ridge regression. An application is made to spline smoothing in nonparametric regression. A counterexample is given to help understand why sometimes GCV may not be asymptotically optimal. The coefficient of variation for the eigenvalues of the information matrix must be large in order to guarantee the optimality of GCV. The proff is based on the connection between…
It is desired to estimate a real valued function F on the unit square having observed F with error at N points in the square. F is assumed to be drawn from a particular Gaussian process and measured with independent Gaussian errors. The proposed estimate is the Bayes estimate of F given the data. The roughness penalty corresponding to the prior is derived and it is shown how the Bayesian technique can be regarded as a generalisation of variance components analysis. The…
Simultaneous confidence intervals for percentiles of the normal regression model similar to those given by Steinhorst and Bowden (1971) are considered. Kanofsky's (1968) confidence band for a single normal distribution is modified and extended to the regression model. The confidence bands that are simultaneous in all percentiles provide corresponding confidence bands for the cumulative conditional normal distribution functions. The various procedures are compared with…
We consider the estimation problem for the parameter ϑ0 of a stationary ARMA (p,q) process, with independent and identically, but not necessary normally distributed errors. First we prove local asymptotic normality (LAN) for this model. Then we construct locally asymptotically minimax (LAM) estimators, which asymptotically achieve the smallest possible covariance matrix. Utilizing these, we finally obtain strongly adaptive estimators, by using usual kernel…
A new sieve estimator for the mean function m(t) of a general Gaussian process of known covariance is presented. The estimator m^(t) is given explicitly from the data and has a simple distribution. It is shown that m^(t) is asymptotically unbiased and consistent (weakly and in mean square) at each t, and that m^ is strongly consistent for m in an appropriate norm. No assumptions are made about the "time" parameter or the covariance.
In order to improve the rate of decrease of the IMSE for nonparametric kernel density estimators with nonrandom bandwidth beyond O(n−4/5) all current methods must relax the constraint that the density estimate be a bona fide function, that is, be nonnegative and integrate to one. In this paper we show how to achieve similar improvement without relaxing any of these constraints. The method can also be applied for orthogonal series, adaptive orthogonal series, spline,…
The Edgeworth Expansion for U-Statistics of Degree Two
An Edgeworth expansion with remainder o(N−1) is established for a U-statistic with a kernel h of degree 2. The assumptions involved appear to be very mild; in particular, the common distribution of the summands h(Xi,Xj) is not assumed to be smooth.
Consider a regression model Yi=xi′β+Ri, i=1,\cdots,n, where Ri are i.i.d. with c.d.f., F; $x_i\inR^p$ and $\beta\inR^p$. Let β^ be a M-estimator defined using kernel, ψ; let F^n(x) denote the empiric distribution of the residuals, Yi−xi′β^ and let F^n∗ be the empiric c.d.f. of the errors, Ri. Under suitable smoothness conditions on ψ, F, and the density F' = f and conditions requiring essentially that xi…
Asymptotic Optimality of CL and Generalized Cross-Validation in Ridge Regression with Application to Spline Smoothing
The asymptotic optimality of Mallows' CL and generalized cross-validation is demonstrated in the setting of ridge regression. An application is made to spline smoothing in nonparametric regression. A counterexample is given to help understand why sometimes GCV may not be asymptotically optimal. The coefficient of variation for the eigenvalues of the information matrix must be large in order to guarantee the optimality of GCV. The proff is based on the connection between…