This paper studies some asymptotic properties of density estimates $\hat{f}$ of $f$ based on $d$-variate delta sequences. The mean-square consistency, almost sure consistency, and asymptotic normality of $\hat{f}$ have been obtained as corollaries to the $L_1$ convergence properties of these delta sequences. Estimators based on kernel functions, orthogonal series, and some histogram methods can be obtained as special cases of $\hat{f}$.
Kernel estimation of $f(0)$ is considered where $f$ is a density in some class $\mathscr{F}$ of $d$-dimensional densities, described in terms of a Taylor series expansion. A sequence of kernels which asymptotically minimizes the maximum mean square error of estimation over $\mathscr{F}$ is given. The shape of the kernel is fixed, the size of the window depends on $f(0)$, and an easily computed estimate is obtained to efficiently adapt the sequence to the unknown value of $f(0)$…
SUMMARY Non-parametric estimates of mixing proportions based on kernel-type density estimators are badly suited to several types of data. They suffer from aberrations due to rounding or truncation of the measurements, and their construction involves the crucial choice of the “window size”, or smoothing parameter. In many circumstances estimators based on the empiric distribution function would be more suitable, and in this paper we investigate their properties. The…
Let $U_N$ be a one sample $U$-statistic with kernel $h$ of degree two, such that $Eh(X_1, X_2) = \vartheta$ and $\operatorname{Var} E\lbrack h(X_1, X_2)\mid X_1 \rbrack > 0$. It is shown that for a studentized $U$-statistic $\sup_x|P(N^{1/2}S^{-1}_N(U_N - \vartheta) \leqslant x) - \Phi(x)| = O(N^{-1/2})$ as $N \rightarrow \infty$, where $N^{-1}S^2_N = 4N^{-1}(N - 1)(N - 2)^{-2}\sum^N_{i=1} \lbrack (N - 1)^{-1} \sum_{j\neq i} h(X_i, X_j) - U_N \rbrack^2$ is the jackknife…
Let $f$ be a density possessing some smoothness properties and let $X_1,\cdots, X_n$ be independent observations from $f$. Some desirable properties of orthogonal series density estimates $f_{n,m,\lambda}$ of $f$ of the form $f_{n,m,\lambda}(t) = \sum^n_{\nu = 1} \frac{\hat{f}_\nu}{(1 + \lambda\nu^{2m})} \phi_\nu(t)$ where $\{\phi_\nu\}$ is an orthonormal sequence and $\hat{f}_\nu = (1/n)\sum^n_{j=1} \phi_\nu(X_j)$ is an estimate of $f_\nu = \int \phi_\nu(t)f(t) dt$, are…
A distribution function is estimated by integrating a kernel estimator of the density. Quantiles are estimated by inverting the estimate of the distribution function. Second-order properties of both these estimators are studied.
Adrian W. Bowman·刊期 1980-12 · 卷 67（3）·Biometrika已编入正刊
分类数据核方法的一致性札记AI 解读
Aitchison & Aitken (1976) described a kernel method of estimating the cell probabilities of a multivariate categorical distribution. The criterion for choosing the smoothing parameter is shown to be a cross-validatory choice, as observed by Titterington (1980), and consistency is demonstrated.
Let $S^d_k$ be the set of $d$th order splines on $\lbrack 0, 1 \rbrack$ having $k$ knots $\xi_1 < \xi_2 \cdots < \xi_k$. We consider the estimation of a sufficiently smooth response function $g$, using $n$ uncorrelated observations, by an element $s$ of $S^d_k$. For large $n$ and $k$ we have discussed the asymptotic behavior of the integrated mean square error (IMSE) for two types of estimators: (i) the least squares estimator and (ii) a bias minimizing estimator. The…
To improve the rate of decrease of integrated mean square error for nonparametric kernel density estimators beyond $0(n^{-\frac{4}{5}}),$ we must relax the constraint that the density estimate be a bonafide density function, that is, be nonnegative and integrate to one. All current methods for kernel (and orthogonal series) estimators relax the nonnegativity constraint. In this paper we show how to achieve similar improvement by relaxing the integral constraint only. This is…
Let $\{X(t), - \infty < t < \infty\}$ be a stationary time series with spectral density function $\phi(\lambda).$ Let $\{t_n\}$ be a stationary Poisson point process on the real line. The existence of consistent estimates of $\phi(\lambda)$ based on the discrete-time observations $\{X(t_n)\}^N_{n = 1},$ when the actual sampling times are not known, has been an open question (Beutler). Using an orthogonal series method, a class of spectral estimates is considered and its…
In this paper, we consider the estimation of isotonic, convex or related functions by means of splines. It is shown that certain classes of isotone or convex functions can be represented as a positive cone embedded in a Hilbert space. Using this representation, we give an existence and characterization theorem for isotonic or convex splines. Two special cases are examined showing the existence of a globally monotone cubic smoothing spline and a globally convex quintic…
Admissible designs for polynomial monospline regression are shown to be a proper subset of the admissible designs for spline regression. Sufficient conditions are given for designs for polynomial monospline regression to be admissible, and examples are used to show that simply counting points in intervals does not determine admissibility for monospline functions.
A nonparametric probability density estimator is proposed that is optimal with respect to a discretized form of a continuous penalized-likelihood criterion functional. Approximation results relating the discrete estimator to the estimate obtained by solving the corresponding infinite-dimensional problem are presented. The discrete estimator is shown to be consistent. The numerical implementation of this discrete estimator is outlined and examples displayed. A simulation…
Let $X_1, \cdots, X_n$ be independent observations with common density $f$. A kernel estimate of the mode is any value of $t$ which maximizes the kernel estimate of the density $f_n$. Conditions are given restricting the density, the kernel, and the bandwidth under which this estimate of the mode has an asymptotic normal distribution. By imposing sufficient restrictions, the rate at which the mean squared error of the estimator converges to zero can be decreased from $n^{-\frac{4}{7}}$…
It is shown that, under some regularity conditions on the kernel, a one-sample $U$-statistic with kernel of degree two admits an asymptotic expansion with remainder term $o(N^{-1})$.
Let $(X, Y)$ be an $\mathbb{R}^d \times \mathbb{R}$-valued random vector and let $(X_1, Y_1), \cdots, (X_n, Y_n)$ be a random sample drawn from its distribution. We study the consistency properties of the kernel estimate $m_n(x)$ of the regression function $m(x) = E\{Y\mid X = x\}$ that is defined by $m_n(x) = \Sigma^n_{i=1} Y_ik((X_i - x)/h_n)/\Sigma^n_{i=1}k((X_i - x)/h_n)$ where $k$ is a bounded nonnegative function on $\mathbb{R}^d$ with compact support and $\{h_n\}$ is…
We develop an objective optimum smoothing procedure for an estimate of the log spectral density, based on smoothing the log periodogram with a smoothing spline. We call the result the optimally smoothed spline (OSS) estimate. We show that an unbiased estimate [Rcirc](λ) of the expected integrated mean square error can be obtained as a function of the smoothing, or “bandwidth” parameter λ. The smoothing parameter then is chosen as that λ that minimizes [Rcirc](λ). The degree…
We develop an objective optimum smoothing procedure for an estimate of the log spectral density, based on smoothing the log periodogram with a smoothing spline. We call the result the optimally smoothed spline (OSS) estimate. We show that an unbiased estimate [Rcirc](λ) of the expected integrated mean square error can be obtained as a function of the smoothing, or “bandwidth” parameter λ. The smoothing parameter then is chosen as that λ that minimizes [Rcirc](λ). The degree…
ADRIAN W. BOWMAN·刊期 1980 · 卷 67（3）·Biometrika已编入正刊
分类数据分析中核方法一致性的一个注记AI 解读
Aitchison & Aitken (1976) described a kernel method of estimating the cell probabilities of a multivariate categorical distribution. The criterion for choosing the smoothing parameter is shown to be a cross-validatory choice, as observed by Titterington (1980), and consistency is demonstrated.
Let $X_1, \cdots, X_n$ be a sequence of independent random vectors taking values in $\mathbf{\mathbb{R}}^d$ with a common probability density $f$. If $f_n(x) = (1/h) h^{-d}_n\sum^n_{i = 1}K((x - X_i)/h_n)$ is the kernel estimate of $f$ from $X_1, \cdots, X_n$ then conditions on $K$ and $\{h_n\}$ are given which insure that $\int|f_n(x) - f(x)|dx \rightarrow_n 0$ in probability or with probability one. No continuity conditions are imposed on $f$.
Consider the realization of the process $y(t) = \Sigma^n_{k=1}\theta_kf_k(t) + \xi(t)$ on the interval $T = \lbrack 0, 1\rbrack$ for functions $f_1(t), f_2(t), \cdots, f_n(t)$ in $H(R)$, the reproducing kernel Hilbert space with reproducing kernel $R(s, t)$ on $T \times T$, where $R(s, t) = E\xi(s)\xi(t)$ is assumed to be continuous and known. Problems of the selection of functions $\{f_k(t)\}^n_{k=1}$ are discussed for $D$-optimal, $A$-optimal and other criteria of optimal…
Bayesian analysis using Monte Carlo integration is a powerful method for univariate inference. This approach makes possible multiparameter flexibility within families of univariate distributions. These distributions are defined in this article by increasing spline functions superimposed on probability paper coordinate systems. Smoothing is controlled by the prior distribution. The prior distribution also can express uncertainties about the form of the tails when…
Let $T \subseteq I$ be sets of real numbers. Let $\{Y(t): t\in I\}$ be a real time series whose covariance kernel is assumed known and positive definite. The mean is assumed either to be known or to be an unknown member of a known class of functions on $I$. For each fixed $s\in I, Y(s)$ is predicted by a minimum mean square error unbiased linear predictor $\hat{Y}(s)$ based on $\{Y(t): t\in T\}$. If $\hat{y}(s)$ is the evaluation of $\hat{Y}(s)$ given that the sample path…
This paper studies some asymptotic properties of density estimates f^ of f based on d-variate delta sequences. The mean-square consistency, almost sure consistency, and asymptotic normality of f^ have been obtained as corollaries to the L1 convergence properties of these delta sequences. Estimators based on kernel functions, orthogonal series, and some histogram methods can be obtained as special cases of f^.
Kernel estimation of f(0) is considered where f is a density in some class F of d-dimensional densities, described in terms of a Taylor series expansion. A sequence of kernels which asymptotically minimizes the maximum mean square error of estimation over F is given. The shape of the kernel is fixed, the size of the window depends on f(0), and an easily computed estimate is obtained to efficiently adapt the sequence to the unknown value of f(0)…
The Order of the Normal Approximation for a Studentized U-Statistic
Let UN be a one sample U-statistic with kernel h of degree two, such that Eh(X1,X2)=ϑ and VarE[h(X1,X2)∣X1]>0. It is shown that for a studentized U-statistic supx∣P(N1/2SN−1(UN−ϑ)⩽x)−Φ(x)∣=O(N−1/2) as N→∞, where N−1SN2=4N−1(N−1)(N−2)−2∑i=1N[(N−1)−1∑j=ih(Xi,Xj)−UN]2 is the jackknife…
Let f be a density possessing some smoothness properties and let X1,⋯,Xn be independent observations from f. Some desirable properties of orthogonal series density estimates fn,m,λ of f of the form fn,m,λ(t)=∑ν=1n(1+λν2m)f^νϕν(t) where {ϕν} is an orthonormal sequence and f^ν=(1/n)∑j=1nϕν(Xj) is an estimate of fν=∫ϕν(t)f(t)dt, are…
Let Skd be the set of dth order splines on [0,1] having k knots ξ1<ξ2⋯<ξk. We consider the estimation of a sufficiently smooth response function g, using n uncorrelated observations, by an element s of Skd. For large n and k we have discussed the asymptotic behavior of the integrated mean square error (IMSE) for two types of estimators: (i) the least squares estimator and (ii) a bias minimizing estimator. The…
To improve the rate of decrease of integrated mean square error for nonparametric kernel density estimators beyond 0(n−54), we must relax the constraint that the density estimate be a bonafide density function, that is, be nonnegative and integrate to one. All current methods for kernel (and orthogonal series) estimators relax the nonnegativity constraint. In this paper we show how to achieve similar improvement by relaxing the integral constraint only. This is…
Let {X(t),−∞<t<∞} be a stationary time series with spectral density function ϕ(λ). Let {tn} be a stationary Poisson point process on the real line. The existence of consistent estimates of ϕ(λ) based on the discrete-time observations {X(tn)}n=1N, when the actual sampling times are not known, has been an open question (Beutler). Using an orthogonal series method, a class of spectral estimates is considered and its…
Let X1,⋯,Xn be independent observations with common density f. A kernel estimate of the mode is any value of t which maximizes the kernel estimate of the density fn. Conditions are given restricting the density, the kernel, and the bandwidth under which this estimate of the mode has an asymptotic normal distribution. By imposing sufficient restrictions, the rate at which the mean squared error of the estimator converges to zero can be decreased from n−74…
An Edgeworth Expansion for U-Statistics
It is shown that, under some regularity conditions on the kernel, a one-sample U-statistic with kernel of degree two admits an asymptotic expansion with remainder term o(N−1).
Let (X,Y) be an Rd×R-valued random vector and let (X1,Y1),⋯,(Xn,Yn) be a random sample drawn from its distribution. We study the consistency properties of the kernel estimate mn(x) of the regression function m(x)=E{Y∣X=x} that is defined by mn(x)=Σi=1nYik((Xi−x)/hn)/Σi=1nk((Xi−x)/hn) where k is a bounded nonnegative function on Rd with compact support and {hn} is…
The L1 Convergence of Kernel Density Estimates
Let X1,⋯,Xn be a sequence of independent random vectors taking values in Rd with a common probability density f. If fn(x)=(1/h)hn−d∑i=1nK((x−Xi)/hn) is the kernel estimate of f from X1,⋯,Xn then conditions on K and {hn} are given which insure that ∫∣fn(x)−f(x)∣dx→n0 in probability or with probability one. No continuity conditions are imposed on f.
Consider the realization of the process y(t)=Σk=1nθkfk(t)+ξ(t) on the interval T=[0,1] for functions f1(t),f2(t),⋯,fn(t) in H(R), the reproducing kernel Hilbert space with reproducing kernel R(s,t) on T×T, where R(s,t)=Eξ(s)ξ(t) is assumed to be continuous and known. Problems of the selection of functions {fk(t)}k=1n are discussed for D-optimal, A-optimal and other criteria of optimal…
Let T⊆I be sets of real numbers. Let {Y(t):t∈I} be a real time series whose covariance kernel is assumed known and positive definite. The mean is assumed either to be known or to be an unknown member of a known class of functions on I. For each fixed s∈I,Y(s) is predicted by a minimum mean square error unbiased linear predictor Y^(s) based on {Y(t):t∈T}. If y^(s) is the evaluation of Y^(s) given that the sample path…