This paper discusses several lower bound results for the asymptotic performance of estimators of smooth functionals in i.i.d. models. The key idea is to look at a set of local limiting distributions of an estimator sequence, rather than to impose regularity conditions, or to consider limits of maximum risk. Special attention is paid to situations where the tangent cone is not a linear space. As an example, the local asymptotic minimax risk in mixture models is computed.
Likelihood regions are shown to be robust in the sense that their posterior probability content is relatively insensitive to contaminations of the prior. This provides a Bayesian interpretation of regions that are commonly used by frequentists to construct confidence intervals and whose use are also advocated by the pure likelihood approach.
Pranab Kumar Sen · Tatsuya Kubokawa · A. K. Md. Ehsanes Saleh·刊期 1989-09-01 · 卷 17(3)·The Annals of Statistics已编入正刊
Pitman 接近性准则意义下的 Stein 悖论AI 解读
The dominance and related optimality properties of the usual Stein-rule or shrinkage estimators are typically developed for quadratic error loss functions. It is shown that under the classical Pitman closeness criterion the Stein-rule estimators possess a similar dominance property when the "closeness" measure is based on suitable quadratic norms.
Consider the classical invariant decision problem of estimating an unknown continuous distribution function $F,$ with the loss function $L(F, a) = \int(F(t) - a(t))^2\lbrack F(t) \rbrack^\alpha \lbrack 1 - F(t) \rbrack^\beta dF(t),$ and a random sample of size $n$ from $F.$ It is proved that the best invariant estimator is inadmissible when: 1. $ n > 0, - 1 < \alpha, \beta \leq 0 \text{and} -1 \leq \alpha + \beta.$ 2. $ n > 0, -1 < \alpha = \beta \leq - \frac{1}{2}.$ 3. $ n > 1, (\mathrm{i}) \alpha = -1 \text{and} \beta = 0, \text{or} (\mathrm{ii}) \alpha = 0 \text{and} \beta = -1.$…
Let $\Omega$ be an interval and let $F_\omega, \omega \in \Omega,$ denote a one-parameter exponential family of probability distributions on $\mathscr{R} = (-\infty, \infty),$ each of which has a finite mean $\theta,$ depending on some unknown parameter $\omega \in \Omega.$ The main results of this paper determine an asymptotic lower bound for the local minimax regret, under a general smooth loss function and for a general class of estimators of $\theta.$ This bound is…
Optimal-partitioning and minimax risk inequalities are obtained for the classification and multi-hypotheses testing problems. Best possible bounds are derived for the minimax risk for location parameter families, based on the tail concentrations and Levy concentrations of the distributions. Special attention is given to continuous distributions with the maximum likelihood ratio property and to symmetric unimodal continuous distributions. Bounds for general (including…
A strategy of randomizing a design that is $L$-optimal for an ideal model is shown to possess a minimax property under a larger, more realistic model. This generalizes a result of Li.
Suppose that real numbers $y_i$ are associated with the units $i = 1, 2, \ldots, N$ of a population $U$ and that the vector $y = (y_1, y_2, \ldots, y_N)$ is known to be an element of the parameter space $\Theta$. The statistician has to select a sample $s \subset U$ of $n$ units and to employ $y_i, i \in s,$ to estimate $\bar{y} = \sum y_i/N.$ We propose to base this decision on an asymptotic version of the minimax principle. The asymptotically minimax principle is applied…
In this paper we consider generalized chi-square goodness-of-fit tests based on increasingly finer partitions (as the sample size increases) for models with location-scale nuisance parameters. The asymptotic distributions are derived both under the null hypothesis and under local alternatives, obtained by taking contamination families of densities between the null hypothesis and fixed alternative hypotheses. If the number of random cells increases to infinity, the…
Least squares estimators of the coefficients of an autoregression of known, finite order are biased to order $1/T$, where $T$ is the sample length, unless the observed time series is generated by a unique model for that order. The coefficients of this special model are the fixed point of a linear mapping defined by the bias of the least squares estimator. Separate results are given for models with known mean and unknown mean. The "fixed point models" for different orders of…
We give sufficient conditions for strong consistency of estimators for the order of general nonstationary autoregressive models based on the minimization of an information criterion a la Akaike's (1969) AIC. The case of a time-dependent error variance is also covered by the analysis. Furthermore, the more general case of regressor selection in stochastic regression models is treated.
Robust nonparametric estimators for regression and autoregression are proposed for $\varphi$- and $\alpha$-mixing processes. Two families of $M$-type robust equivariant estimators are considered: (i) estimators based on kernel methods and (ii) estimators based on $k$-nearest neighbor kernel methods. Strong consistency of both families is proved under mild conditions. For the first class the result is true under no assumptions whatsoever on the distribution of the…
The notion of a bootstrap estimator of the distribution of the maximum likelihood estimator in log-linear models is defined for common sampling models. It is shown that the bootstrap estimator is consistent under assumptions which allow the dimension of the model to increase to infinity. Such an approach allows treatment of large, sparse contingency tables.
Athreya showed that the bootstrap distribution of a sum of infinite variance random variables did not (with probability 1) tend weakly to a fixed distribution but instead tended in distribution to a random distribution. In this paper, we give a different proof of Athreya's result motivated by a heuristic large sample representation of the bootstrap distribution.
We consider a class of nonparametric regression estimates introduced by Beran to estimate conditional survival functions in the presence of right censoring. An exponential probability bound for the tails of distributions of kernel estimates of conditional survival functions is derived. This inequality is next used to prove weak and strong uniform consistency results. The developments rest on sharp exponential bounds for the oscillation modulus of multivariate empirical…
Statistical models are considered for (partial) observation of independent, identically distributed failure times, subjected to censoring from the right. A minimal class of censoring patterns is determined under the assumption that the failure intensities are as they would have been without censoring. This class is then used to discuss the statistical models for right censored survival data, where, e.g., the Kaplan-Meier estimator exploits all available information about the…
This paper deals with the distribution of $\mathbf{X} = \sum^{1/2}\mathbf{Z}$, where $\mathbf{Z}: p \times 1$ is distributed as $N_p(0, I_p), \sum$ is a positive definite random matrix and $\mathbf{Z}$ and $\sum$ are independent. Assuming that $\sum = I_p + BB'$, we obtain an asymptotic expansion of the distribution function of $\mathbf{X}$ and its error bound, which is useful in the situation where $\sum$ tends to $I_p$. A stronger version of the expansion is also given.…
An Edgeworth expansion of first order is established for general linear rank statistics under the null hypothesis with a remainder term that is usually of order $n^{-1}$. Furthermore, corresponding results for the second order are formulated, but not proved here. The proof for the first order is based on Stein's method and on an extension of a combinatorial method of Bolthausen. It is also shown that conditions of van Zwet imply up to a small factor our conditions for the…
In sequentially designed experiments with linear models, each design variable may depend on previous responses. The use of such sequential designs does not affect the likelihood function or the functional form of the maximum likelihood estimator, but it may affect sampling distributions. In this paper, asymptotic expansions for sampling distributions are obtained. The expansions are very weak ones in which a confidence curve (a function of the unknown parameters) is replaced…
Michel Broniatowski · Paul Deheuvels · Luc Devroye·刊期 1989-09-01 · 卷 17(3)·The Annals of Statistics已编入正刊
极端次序统计量稳定性与极大似然核密度估计收敛的关系AI 解读
Let $f$ be a density on the real line and let $f_n$ be the kernel estimate of $f$ in which the smoothing factor is obtained by maximizing the cross-validated likelihood product according to the method of Duin and Habbema, Hermans and Vandenbroek. Under mild regularity conditions on the kernel and $f$, we show, among other things that $\int|f_n - f| \rightarrow 0$ almost surely if and only if the sample extremes of $f$ are strongly stable.
It is shown that consistent estimates of the optimal bandwidths for kernel estimators of location and size of a peak of a regression function are available. Such estimates yield the same joint asymptotic distribution of location and size of a peak as the optimal bandwidths themselves. Therefore data-adaptive efficient estimation of peaks is possible. In order to prove this result, the weak convergence of a two-dimensional stochastic process with appropriately scaled…
We study the behavior of regression analysis when there might be some violation of the assumed link function, the functional form of the model which relates the outcome variable $y$ to the regressor variable $\mathbf{x}$ and the random error. We allow the true link function to be completely arbitrary, except that $y$ depends on $\mathbf{x}$ only through a linear combination $\beta\mathbf{x}$. The slope vector $\beta$ is identified only up to a multiplicative scalar. Under…
The problem of constructing honest confidence regions for nonparametric regression is considered. A lower rate of convergence, $n^{-1/4}$, for the size of the confidence region is established. The achievability of this rate is demonstrated using Stein's estimates and the associated unbiased risk estimates. Practical implications are discussed.
Consider a continuous Gaussian random field $z(x)$ defined on a compact set $R \subset \mathbb{R}^d$ with covariance function of the form $\operatorname{cov}(z(x), z(x')) = \sum^k_{i = 1}\theta_iK_i(x,x')$, where the $K_i$'s are specified and $\theta = (\theta_1, \ldots, \theta_k)'$ is to be estimated. Let $\{x_l\}^\infty_{l = 1}$ be a sequence of distinct points in $R$. Based on $z(x_1), \ldots, z(x_N)$, minimum norm quadratic estimation can be used to estimate $\theta$.…
Consider the classical invariant decision problem of estimating an unknown continuous distribution function F, with the loss function L(F,a)=∫(F(t)−a(t))2[F(t)]α[1−F(t)]βdF(t), and a random sample of size n from F. It is proved that the best invariant estimator is inadmissible when: 1. n>0,−1<α,β≤0and−1≤α+β. 2. n>0,−1<α=β≤−21. 3. n>1,(i)α=−1andβ=0,or(ii)α=0andβ=−1.…
Let Ω be an interval and let Fω,ω∈Ω, denote a one-parameter exponential family of probability distributions on R=(−∞,∞), each of which has a finite mean θ, depending on some unknown parameter ω∈Ω. The main results of this paper determine an asymptotic lower bound for the local minimax regret, under a general smooth loss function and for a general class of estimators of θ. This bound is…
A strategy of randomizing a design that is L-optimal for an ideal model is shown to possess a minimax property under a larger, more realistic model. This generalizes a result of Li.
Suppose that real numbers yi are associated with the units i=1,2,…,N of a population U and that the vector y=(y1,y2,…,yN) is known to be an element of the parameter space Θ. The statistician has to select a sample s⊂U of n units and to employ yi,i∈s, to estimate yˉ=∑yi/N. We propose to base this decision on an asymptotic version of the minimax principle. The asymptotically minimax principle is applied…
Least squares estimators of the coefficients of an autoregression of known, finite order are biased to order 1/T, where T is the sample length, unless the observed time series is generated by a unique model for that order. The coefficients of this special model are the fixed point of a linear mapping defined by the bias of the least squares estimator. Separate results are given for models with known mean and unknown mean. The "fixed point models" for different orders of…
Robust nonparametric estimators for regression and autoregression are proposed for φ- and α-mixing processes. Two families of M-type robust equivariant estimators are considered: (i) estimators based on kernel methods and (ii) estimators based on k-nearest neighbor kernel methods. Strong consistency of both families is proved under mild conditions. For the first class the result is true under no assumptions whatsoever on the distribution of the…
This paper deals with the distribution of X=∑1/2Z, where Z:p×1 is distributed as Np(0,Ip),∑ is a positive definite random matrix and Z and ∑ are independent. Assuming that ∑=Ip+BB′, we obtain an asymptotic expansion of the distribution function of X and its error bound, which is useful in the situation where ∑ tends to Ip. A stronger version of the expansion is also given.…
An Edgeworth expansion of first order is established for general linear rank statistics under the null hypothesis with a remainder term that is usually of order n−1. Furthermore, corresponding results for the second order are formulated, but not proved here. The proof for the first order is based on Stein's method and on an extension of a combinatorial method of Bolthausen. It is also shown that conditions of van Zwet imply up to a small factor our conditions for the…
Let f be a density on the real line and let fn be the kernel estimate of f in which the smoothing factor is obtained by maximizing the cross-validated likelihood product according to the method of Duin and Habbema, Hermans and Vandenbroek. Under mild regularity conditions on the kernel and f, we show, among other things that ∫∣fn−f∣→0 almost surely if and only if the sample extremes of f are strongly stable.
We study the behavior of regression analysis when there might be some violation of the assumed link function, the functional form of the model which relates the outcome variable y to the regressor variable x and the random error. We allow the true link function to be completely arbitrary, except that y depends on x only through a linear combination βx. The slope vector β is identified only up to a multiplicative scalar. Under…
The problem of constructing honest confidence regions for nonparametric regression is considered. A lower rate of convergence, n−1/4, for the size of the confidence region is established. The achievability of this rate is demonstrated using Stein's estimates and the associated unbiased risk estimates. Practical implications are discussed.
Consider a continuous Gaussian random field z(x) defined on a compact set R⊂Rd with covariance function of the form cov(z(x),z(x′))=∑i=1kθiKi(x,x′), where the Ki's are specified and θ=(θ1,…,θk)′ is to be estimated. Let {xl}l=1∞ be a sequence of distinct points in R. Based on z(x1),…,z(xN), minimum norm quadratic estimation can be used to estimate θ.…