For each $t$ in an index set $T$, let $P_t$ be a probability measure with mode $M(t)$. In this paper we consider a maximum likelihood nonparametric estimator $\hat{M}_n(t)$ of $M(t)$ subject to the constraint that $\hat{M}_n(\cdot)$ be isotonic with respect to an order on $T$. The estimator is a solution to a minimization problem with zero-one loss. The estimator is not a max-min or min-max representation of "naive" estimators. Naive modal estimators are used but they are…
Asymptotic expansions are derived for the densities of multivariate $M$-estimates. The expansion is based on a saddlepoint technique and yields good accuracy in the tails for small sample sizes. Numerical results are given for robust estimation of location and scale and these are compared with known Monte Carlo results.
We simplify a general heuristic necessary condition of Stein's for adaptive estimation of a Euclidean parameter in the presence of an infinite dimensional shape nuisance parameter and other Euclidean nuisance parameters. We derive sufficient conditions and apply them in the construction of adaptive estimates for the parameters of linear models and multivariate elliptic distributions. We conclude with a review of issues in adaptive estimation.
Hong-Zhi An · Chen Zhao-Guo · E. J. Hannan·刊期 1982-09-01 · 卷 10(3)·The Annals of Statistics已编入正刊
自相关、自回归与自回归逼近AI 解读
Theorems are proved relating to the rate of almost sure convergence of autocovariances, and hence autocorrelations, to their true values. These rates are uniform in the lag up to some order $P(T)$, increasing with $T$. The key assumption is that the process is stationary and the best linear predictor is the best predictor. In particular for an ARMA process and $P(T) = O\{(\ln T)^a\}, a < \infty$, the rate is $O\{(\ln \ln T/T)^{1/2}\}$. These results are used to discuss…
Let $\{X_i\}$ be a sample from $F(x - \theta)$ where $F$ is in a class $\mathscr{F}$ of symmetric distributions on the line and $\theta$ is the location parameter to be estimated. Huber has shown that maximum likelihood estimation has a minimax property over a convex $\mathscr{F}$. Here a simple convex $\mathscr{F}$ is given for which neither $L$- nor $R$-estimation has the minimax property. In particular, this example shows that a recent assertion concerning $L$-estimation…
It is shown that if the density $f(\mathbf{x})$ of $\mathbf{X} = (X_1, \cdots, X_n)$ is Schur-concave, then (1) $P(|X_i| \leq a_i, i = 1, \cdots, n)$ is a Schur-concave function of $(\phi(a_1), \cdots, \phi(a_n))$, and (2) $P\{\Sigma(X_i/a_i)^2 \leq 1\}$ is a Schur-concave function of $(\phi(a^2_1), \cdots, \phi(a^2_n))$, where $\phi; \lbrack 0, \infty) \rightarrow \lbrack 0, \infty)$ is any increasing and convex function. By letting $\phi(a) = a$, (1) implies that $P(|X_i| \leq a_i, i = 1, \cdots, n) \leq P(|X_i| \leq \bar{a}, i = 1, \cdots, n)$…
If a strictly stationary process $\{Z_k\}$ has residuals $Z_{k+1} - \sum^k_{j=1} a_{k,j}Z_j$ independent of $(Z_1, \cdots, Z_k)$ for all $k \geq m$, it is shown that the process is Gaussian or degenerate or $m$-step Markovian. Generalized (nonlinear) autoregressive stationary processes are defined and partially characterized.
For any decision problem, one wishes to find that estimator which minimizes the expected loss. If the loss function is squared error, then the estimator is the mean of the Bayes posterior distribution. Unfortunately the prior distribution may be unknown, but in certain situations empirical Bayes methods can circumvent this problem by using past observations to estimate either the prior or the Bayes estimate directly. Empirical Bayes methods are particularly appealing when…
The asymptotic properties of a family of minimum quantile function distance estimators are considered. These procedures take as the parameter estimates that vector which minimizes a weighted $L^2$ distance between the empirical quantile function and an assumed parametric family of quantile functions. Regularity conditions needed for these estimators to be consistent and asymptotically normal are presented. For single parameter families of distributions, the optimal form of…
We derive the nonparametric maximum likelihood estimate, $\hat{F}$ say, of a lifetime distribution $F$ on the basis of two independent samples, one a sample of size $m$ from $F$ and the other a sample of size $n$ from the length-biased distribution of $F$, i.e. from $G_F(x) = \int^x_0 u dF(u)/\mu, \mu = \int^\infty_0 x dF(x)$. We further show that $(m + n)^{1/2}(\hat{F} - F)$ converges weakly to a pinned Gaussian process with a simple covariance function, when $m + n \rightarrow \infty$…
This paper is concerned with the theory of unbiased estimation for nonparametric families of distributions subject to "generalised moment" restrictions. Necessary and sufficient conditions under which symmetric statistics are unique minimum variance unbiased estimators of their expectations are obtained, and some new boundedly complete families of distributions are exhibited.
The product limit estimator due to Kaplan and Meier (1958) is well-known to be the nonparametric maximum likelihood estimator of a distribution function based on censored data. It is shown here that the product limit estimator is an asymptotically optimal estimator in two senses: in the sense of a Hajek-Beran type representation theorem for regular estimators; and in an asymptotic minimax sense similar to the classical result for the uncensored case due to Dvoretzky, Kiefer,…
In 1952 Chernoff introduced a measure of asymptotic efficiency for tests. Comparison in the sense of Chernoff is concerned with fixed alternatives. In contrast to Bahadur's approach, where the probabilities of first and second kind are treated in an unbalanced way, in Chernoff's approach both probabilities go to zero. For the calculation of Chernoff efficiencies one has to develop large deviation theorems both under the null hypothesis and under the alternative hypothesis.…
The concept of $H_0$-contiguity is studied for certain nonparametric testing problems. Furthermore, it is shown that a meaningful sample definition of Pitman efficiency is possible under $H_0$-contiguity, and this definition turns out to coincide with the usual efficiency concept based on comparing asymptotic distributions.
The two-sample problem of testing against location shift is fundamental to much of the theory of rank tests. Generally testing and estimation is carried out with a fixed (non-random) set of scores for the ranks. However Beran (1974), following ideas of Stein and Hajek, developed a notable class of adaptive estimators. When used in testing, these give asymptotically efficient tests, regardless of the underlying distribution. These ideas are used here to focus attention upon…
We propose a method of estimating the endpoint, $\theta$, of a distribution when only limited information is available about the behaviour of the distribution in the neighbourhood of $\theta$. By using increasing numbers of extreme order statistics we obtain an estimator which improves on earlier estimators based on only a bounded number of extremes. In a certain particular model our estimator is equal to a maximum likelihood estimator, but it is robust against departures…
This paper describes how sufficiency and invariance considerations can be applied in problems of confidence set estimation to reduce the class of set estimators under investigation. Let $X$ be a random variable taking values in $\mathscr{X}$ with distribution $P_\theta, \theta \in \Theta$, and suppose a confidence set is desired for $\gamma = \gamma(\theta)$, where $\gamma$ takes values in $\Gamma$. The main tools used are (i) the representation of randomized set estimators…
An estimation procedure is presented for the coefficients of the nonlinear functional relation, where observations are subject to measurement error. The distributional properties of the estimators are derived, and a consistent estimator of the covariance matrix is given. In deriving the results it is assumed that the covariance matrix of the observational errors is known and that this covariance matrix is $o(n^{-1/3})$, where $n$ is the index of the sequence of estimators.
The model $E(y \mid x) = \theta_0 + \sum^k_{i=1} \theta_ix_i + \psi(\mathbf{x})$ is considered, where $\psi(\mathbf{x})$ is an unknown contamination with $| \psi(\mathbf{x})|$ bounded by given $\varphi(\mathbf{x})$. Optimal designs are studied in terms of least squares estimation and a family of minimax criteria. In particular, analogs of D-, A- and E-optimal designs are studied in the general case of an arbitrary $k$. Some commonly used integer designs are considered and…
In a variety of statistical problems, one is interested in the limiting distribution of statistics $\hat{T}_n = T_n(y_1, y_2, \cdots, y_n; \hat{\lambda}_n)$, where $\hat{\lambda}_n$ is an estimator of a parameter in the distribution of the $y_i$ and where the limiting distribution of $T_n = T_n(y_1, y_2, \cdots, y_n; \lambda)$ is relatively easy to find. For cases in which the limiting distribution of $T_n$ is normal with mean independent of $\lambda$, a useful method is…
Often a statistic of interest would take the form of a member of a common family, except that some vital parameter is unknown and must be estimated. This paper describes methods for showing the asymptotic normality of such statistics with estimated parameters. Whether or not the limiting distribution is affected by the estimator is primarily a question of whether or not the limiting mean (derived by replacing the estimator by a mathematical variable) has a nonzero derivative…
If $M$ is not convex, and the underlying density is multi-modal, $M$-estimators can be inconsistent. Examples are given, as well as some positive results.
Stochastic approximation methods are used to generate a sequence of "$M$-estimates" for the unknown parameters of an autoregressive process of known, finite order which may have heavy-tailed innovations. Weak dependence properties, which can be demonstrated for many autoregressive processes, are used in the proof that the sequence converges almost surely to the parameters. A brief Monte Carlo study verifies that bounded influence functions provide protection for recursive…
We consider a heteroscedastic linear model in which the variances are given by a parametric function of the mean responses and a parameter $\theta$. We propose robust estimates for the regression parameter $\beta$ and show that, as long as a reasonable starting estimate of $\theta$ is available, our estimates of $\beta$ are asymptotically equivalent to the natural estimate obtained with known variances. A particular method for estimating $\theta$ is proposed and shown by…
For each t in an index set T, let Pt be a probability measure with mode M(t). In this paper we consider a maximum likelihood nonparametric estimator M^n(t) of M(t) subject to the constraint that M^n(⋅) be isotonic with respect to an order on T. The estimator is a solution to a minimization problem with zero-one loss. The estimator is not a max-min or min-max representation of "naive" estimators. Naive modal estimators are used but they are…
Small Sample Asymptotic Expansions for Multivariate M-Estimates
Asymptotic expansions are derived for the densities of multivariate M-estimates. The expansion is based on a saddlepoint technique and yields good accuracy in the tails for small sample sizes. Numerical results are given for robust estimation of location and scale and these are compared with known Monte Carlo results.
Theorems are proved relating to the rate of almost sure convergence of autocovariances, and hence autocorrelations, to their true values. These rates are uniform in the lag up to some order P(T), increasing with T. The key assumption is that the process is stationary and the best linear predictor is the best predictor. In particular for an ARMA process and P(T)=O{(lnT)a},a<∞, the rate is O{(lnlnT/T)1/2}. These results are used to discuss…
L- and R-Estimation and the Minimax Property
Let {Xi} be a sample from F(x−θ) where F is in a class F of symmetric distributions on the line and θ is the location parameter to be estimated. Huber has shown that maximum likelihood estimation has a minimax property over a convex F. Here a simple convex F is given for which neither L- nor R-estimation has the minimax property. In particular, this example shows that a recent assertion concerning L-estimation…
It is shown that if the density f(x) of X=(X1,⋯,Xn) is Schur-concave, then (1) P(∣Xi∣≤ai,i=1,⋯,n) is a Schur-concave function of (ϕ(a1),⋯,ϕ(an)), and (2) P{Σ(Xi/ai)2≤1} is a Schur-concave function of (ϕ(a12),⋯,ϕ(an2)), where ϕ;[0,∞)→[0,∞) is any increasing and convex function. By letting ϕ(a)=a, (1) implies that P(∣Xi∣≤ai,i=1,⋯,n)≤P(∣Xi∣≤aˉ,i=1,⋯,n)…
If a strictly stationary process {Zk} has residuals Zk+1−∑j=1kak,jZj independent of (Z1,⋯,Zk) for all k≥m, it is shown that the process is Gaussian or degenerate or m-step Markovian. Generalized (nonlinear) autoregressive stationary processes are defined and partially characterized.
Asymptotic Properties of Weighted L2 Quantile Distance Estimators
The asymptotic properties of a family of minimum quantile function distance estimators are considered. These procedures take as the parameter estimates that vector which minimizes a weighted L2 distance between the empirical quantile function and an assumed parametric family of quantile functions. Regularity conditions needed for these estimators to be consistent and asymptotically normal are presented. For single parameter families of distributions, the optimal form of…
We derive the nonparametric maximum likelihood estimate, F^ say, of a lifetime distribution F on the basis of two independent samples, one a sample of size m from F and the other a sample of size n from the length-biased distribution of F, i.e. from GF(x)=∫0xudF(u)/μ,μ=∫0∞xdF(x). We further show that (m+n)1/2(F^−F) converges weakly to a pinned Gaussian process with a simple covariance function, when m+n→∞…
H0-Contiguity in Nonparametric Testing Problems and Sample Pitman Efficiency
The concept of H0-contiguity is studied for certain nonparametric testing problems. Furthermore, it is shown that a meaningful sample definition of Pitman efficiency is possible under H0-contiguity, and this definition turns out to coincide with the usual efficiency concept based on comparing asymptotic distributions.
We propose a method of estimating the endpoint, θ, of a distribution when only limited information is available about the behaviour of the distribution in the neighbourhood of θ. By using increasing numbers of extreme order statistics we obtain an estimator which improves on earlier estimators based on only a bounded number of extremes. In a certain particular model our estimator is equal to a maximum likelihood estimator, but it is robust against departures…
This paper describes how sufficiency and invariance considerations can be applied in problems of confidence set estimation to reduce the class of set estimators under investigation. Let X be a random variable taking values in X with distribution Pθ,θ∈Θ, and suppose a confidence set is desired for γ=γ(θ), where γ takes values in Γ. The main tools used are (i) the representation of randomized set estimators…
An estimation procedure is presented for the coefficients of the nonlinear functional relation, where observations are subject to measurement error. The distributional properties of the estimators are derived, and a consistent estimator of the covariance matrix is given. In deriving the results it is assumed that the covariance matrix of the observational errors is known and that this covariance matrix is o(n−1/3), where n is the index of the sequence of estimators.
Optimal Robust Designs: Linear Regression in Rk
The model E(y∣x)=θ0+∑i=1kθixi+ψ(x) is considered, where ψ(x) is an unknown contamination with ∣ψ(x)∣ bounded by given φ(x). Optimal designs are studied in terms of least squares estimation and a family of minimax criteria. In particular, analogs of D-, A- and E-optimal designs are studied in the general case of an arbitrary k. Some commonly used integer designs are considered and…
In a variety of statistical problems, one is interested in the limiting distribution of statistics T^n=Tn(y1,y2,⋯,yn;λ^n), where λ^n is an estimator of a parameter in the distribution of the yi and where the limiting distribution of Tn=Tn(y1,y2,⋯,yn;λ) is relatively easy to find. For cases in which the limiting distribution of Tn is normal with mean independent of λ, a useful method is…
On Inconsistent M-Estimators
If M is not convex, and the underlying density is multi-modal, M-estimators can be inconsistent. Examples are given, as well as some positive results.
Recursive Computation of M-Estimates for the Parameters of a Finite Autoregressive Process
Stochastic approximation methods are used to generate a sequence of "M-estimates" for the unknown parameters of an autoregressive process of known, finite order which may have heavy-tailed innovations. Weak dependence properties, which can be demonstrated for many autoregressive processes, are used in the proof that the sequence converges almost surely to the parameters. A brief Monte Carlo study verifies that bounded influence functions provide protection for recursive…
We consider a heteroscedastic linear model in which the variances are given by a parametric function of the mean responses and a parameter θ. We propose robust estimates for the regression parameter β and show that, as long as a reasonable starting estimate of θ is available, our estimates of β are asymptotically equivalent to the natural estimate obtained with known variances. A particular method for estimating θ is proposed and shown by…