We study the asymptotic properties of estimators of the tail of a distribution based on the excesses over a threshold. A key idea is the use of Pickands' generalised Pareto distribution and its fitting, in most cases, by the method of maximum likelihood. The results cover all three limiting types of extreme value theory. We propose a new estimator for an index of regular variation and show that it often performs better than Hill's estimator. We give new results for…
We establish rates for strong approximations of the bootstrapped product-limit process and the corresponding quantile process. These results are used to show weak convergence of bootstrapped total time on test and Lorenz curve processes to the same limiting Gaussian processes as for the unbootstrapped versions. We develop fully nonparametric confidence bands and tests for these curves and apply these results to prostate cancer. We also present almost sure results for the…
Stochastic procedures are randomized tests, estimates and confidence sets with two properties: (i) They are functions of an original sample and one or more artificially constructed auxiliary samples. (ii) They become nearly nonrandomized when the auxiliary samples increase in size. The stochastic procedures of this paper, which arise as approximations to numerically intractable procedures, involve iterated bootstrap techniques and random sampling schemes over abstract…
A class of simple adaptive allocation rules is proposed for the problem (often called the "multi-armed bandit problem") of sampling $x_1, \cdots x_N$ sequentially from $k$ populations with densities belonging to an exponential family, in order to maximize the expected value of the sum $S_N = x_1 + \cdots + x_N$. These allocation rules are based on certain upper confidence bounds, which are developed from boundary crossing theory, for the $k$ population parameters. The rules…
This paper considers the problem of sequential point and fixed-width confidence interval estimation of the location parameter when the errors form an autoregressive process with unknown scale and autoregressive parameters. The sequential point estimator considered here is based on sample mean and is shown to be asymptotically risk efficient as the cost per observation tends to zero. The sequential interval estimator is shown to be asymptotically consistent and the…
We show that the empirical quantile process from an ARMA$(1, q)$ process which is strongly mixing $\Delta_s$, and is either Gaussian or double exponential, converges to a Gaussian process. This result is used to derive model-free one-step-ahead prediction intervals for such processes. Simulations demonstrate where the asymptotic theory can and cannot be applied to small samples.
A first-order autoregressive process, $Y_t = \beta Y_{t - 1} + \epsilon_t$, is said to be nearly nonstationary when $\beta$ is close to one. The limiting distribution of the least-squares estimate $b_n$ for $\beta$ is studied when $Y_t$ is nearly nonstationary. By reparameterizing $\beta$ to be $1 - \gamma/n, \gamma$ being a fixed constant, it is shown that the limiting distribution of $\tau_n = (\sum^n_{t = 1}Y^2_{t - 1})^{1/2}(b_n - \beta)$ converges to $\mathscr{L}(\gamma)$…
Elaborating on the work of Ibragimov and Has'minskii (1981) we prove a law of large deviations (LLD) for $M$-estimators, i.e., those estimators which maximize a functional, continuous in the parameter, of the observations. This LLD is applied, using the results of Petrov (1975), to the problem of parametrical nonlinear regression in the situation of discrete time, independent errors and regression functions which are continuous in the parameter. This improves a result of…
The purpose of this paper is to examine the properties of the histogram when the cells are allowed to be arbitrary. Given a random sample from an unknown probability density $f$ on $I$, we wish to construct a histogram. Any partition of $I$ can be used as cells. The optimal partition minimizes the mean integrated squared error (MISE) of the histogram from $f$. An expression is found for the infimum of MISE over all partitions. It is proved that the infimum is attained…
Let us consider the class of all unimodal densities defined on some interval of length $L$ and bounded by $H$; we shall study the minimax risk over this class, when we estimate using $n$ i.i.d. observations, the loss being measured by the $\mathbb{L}^1$ distance between the estimator and the true density. We shall prove that if $S = \operatorname{Log}(HL + 1)$, upper and lower bounds for the risk are of the form $C(S/n)^{1/3}$ and the ratio between those bounds is smaller…
In both density and regression estimation problems, the $k$-nearest neighbor estimators with $k$ varying in an appropriate range, when transformed to continuous time stochastic processes, are shown to have a common limiting structure under the usual second-order smoothness conditions as the sample size tends to $\infty$. These results lead to asymptotic linear models in which BLUE's and suitably biased linear combinations are considered.
$C_p, C_L$, cross-validation and generalized cross-validation are useful data-driven techniques for selecting a good estimate from a proposed class of linear estimates. The asymptotic behaviors of these procedures are studied. Some easily interpretable conditions are derived to demonstrate the asymptotic optimality. It is argued that cross-validation and generalized cross-validation can be viewed as some special ways of applying $C_L$. Applications in nearest-neighbor…
The paper gives a contribution to the problem of finding optimal linear regression designs, when only $s$ out of $k$ regression parameters are to be estimated. Also, a treatment of design admissibility for the parameters of interest is included. Previous results of Kiefer and Wolfowitz (1959), Karlin and Studden (1966) and Atwood (1969) are generalized. In particular, a connection to Tchebycheff-type approximation of $\mathbb{R}^s$-valued functions is found, which has been…
The Efron-Stein inequality is applied to prove that the kernel density estimate $f_n$, with an arbitrary nonnegative kernel and an arbitrary smoothing factor, satisfies the inequality $\operatorname{var}(\int|f_n - f|) \leq 4/n$ for all densities $f$. Similar inequalities are obtained for other estimates.
It is shown under appropriate conditions that Rousseeuw's minimum volume estimator and other $S$-estimators of multivariate location and dispersion parameters are consistent. Under certain differentiability conditions the estimates are asymptotically normally distributed with a norming factor of $n^{1/2}$.
Herein we study a multivariate version of the adaptive stochastic approximation developed recently by Lai and Robbins. An adaptive procedure which involves a Venter-type estimate of the Jacobian of the response function is proposed and shown to be asymptotically efficient from both the estimation and the control points of view.
We generalize the projection pursuit procedure of Friedman and Stuetzle (abstract version) and prove strong convergence. This answers a question of Huber.
An empirical Bayes test for testing $\vartheta \leq \vartheta_0$ against $\vartheta > \vartheta_0$ for the uniform distribution on $\lbrack 0, \vartheta)$ is discussed. The relation is shown with the estimation of a decreasing density on $\lbrack 0, \infty)$ and a monotone empirical Bayes test is derived based on the least-concave majorant of the empirical distribution function. The asymptotic distribution of the Bayes risk is obtained and some Monte Carlo results are given.
The main concerns of this paper are the definition of coherent inference (dF-coherent inference) in conformity with a theory of conditional probabilities derived from de Finetti's coherence principle, and a critical comparison of such a definition with one proposed by Heath, Lane and Sudderth.
Malay Ghosh · David M. Nickerson · Pranab K. Sen·刊期 1987-06-01 · 卷 15（2）·The Annals of Statistics已编入正刊
序贯收缩估计AI 解读
The paper develops a class of James-Stein estimators that dominates the sample mean under sequential sampling schemes of Ghosh, Sinha and Mukhopadhyay (1976). Asymptotic risk expansions of the sample mean and James-Stein estimators are provided up to the second-order term. Also, a Monte Carlo study is undertaken to compare the risks of these estimators.
Let $X_i, i = 1, 2, \ldots, k$, be independent random variables distributed according to a one-parameter exponential family with parameter $\theta_i$. Assume also that the probability density function of $X_i$ is a Polya frequency function of order two $(PF_2)$. Consider the null hypothesis $H_0: \theta_1 = \theta_2 = \cdots = \theta_k$ against the alternative $K$: not $H_0$. We show that any permutation invariant test of size $\alpha$, whose conditional (on $T = \sum^k_{i = 1}X_i)$…
In statistical decision theory, an important question is to characterize the admissible rules. In this paper, we establish complete class theorems for estimating the noncentrality parameter of noncentral chi-square and noncentral $F$ distributions under squared error loss. Under a minor assumption, any admissible estimator must be a generalized Bayes rule. Using this result, we prove that the positive part of the UMVUE is inadmissible.
The present paper considers inference for the statistical model specifying proportionality between independent Wishart distributions. It is shown that the maximum likelihood estimate of the common covariance matrix and the constants of proportionality is the unique solution to the likelihood equation and an iterative procedure determining the estimator is given. The model is an exponential transformation model, which implies the existence of an exact ancillary (a maximal…
Let $X_1, X_2, \ldots, X_n$ be independent identically distributed random variables with $EX^2_1 = \infty$ but $X_1$ belonging to the domain of attraction of a stable law. It is known that the sample mean $\bar{X}_n$ appropriately normalized converges to a stable law. It is shown here that the bootstrap version of the normalized mean has a random distribution (given the sample) whose limit is also a random distribution implying that the naive bootstrap could fail in the…
A class of simple adaptive allocation rules is proposed for the problem (often called the "multi-armed bandit problem") of sampling x1,⋯xN sequentially from k populations with densities belonging to an exponential family, in order to maximize the expected value of the sum SN=x1+⋯+xN. These allocation rules are based on certain upper confidence bounds, which are developed from boundary crossing theory, for the k population parameters. The rules…
We show that the empirical quantile process from an ARMA(1,q) process which is strongly mixing Δs, and is either Gaussian or double exponential, converges to a Gaussian process. This result is used to derive model-free one-step-ahead prediction intervals for such processes. Simulations demonstrate where the asymptotic theory can and cannot be applied to small samples.
A first-order autoregressive process, Yt=βYt−1+ϵt, is said to be nearly nonstationary when β is close to one. The limiting distribution of the least-squares estimate bn for β is studied when Yt is nearly nonstationary. By reparameterizing β to be 1−γ/n,γ being a fixed constant, it is shown that the limiting distribution of τn=(∑t=1nYt−12)1/2(bn−β) converges to L(γ)…
Elaborating on the work of Ibragimov and Has'minskii (1981) we prove a law of large deviations (LLD) for M-estimators, i.e., those estimators which maximize a functional, continuous in the parameter, of the observations. This LLD is applied, using the results of Petrov (1975), to the problem of parametrical nonlinear regression in the situation of discrete time, independent errors and regression functions which are continuous in the parameter. This improves a result of…
The purpose of this paper is to examine the properties of the histogram when the cells are allowed to be arbitrary. Given a random sample from an unknown probability density f on I, we wish to construct a histogram. Any partition of I can be used as cells. The optimal partition minimizes the mean integrated squared error (MISE) of the histogram from f. An expression is found for the infimum of MISE over all partitions. It is proved that the infimum is attained…
Let us consider the class of all unimodal densities defined on some interval of length L and bounded by H; we shall study the minimax risk over this class, when we estimate using n i.i.d. observations, the loss being measured by the L1 distance between the estimator and the true density. We shall prove that if S=Log(HL+1), upper and lower bounds for the risk are of the form C(S/n)1/3 and the ratio between those bounds is smaller…
Weak Convergence of k-NN Density and Regression Estimators with Varying k and Applications
In both density and regression estimation problems, the k-nearest neighbor estimators with k varying in an appropriate range, when transformed to continuous time stochastic processes, are shown to have a common limiting structure under the usual second-order smoothness conditions as the sample size tends to ∞. These results lead to asymptotic linear models in which BLUE's and suitably biased linear combinations are considered.
Asymptotic Optimality for Cp,CL, Cross-Validation and Generalized Cross-Validation: Discrete Index Set
Cp,CL, cross-validation and generalized cross-validation are useful data-driven techniques for selecting a good estimate from a proposed class of linear estimates. The asymptotic behaviors of these procedures are studied. Some easily interpretable conditions are derived to demonstrate the asymptotic optimality. It is argued that cross-validation and generalized cross-validation can be viewed as some special ways of applying CL. Applications in nearest-neighbor…
The paper gives a contribution to the problem of finding optimal linear regression designs, when only s out of k regression parameters are to be estimated. Also, a treatment of design admissibility for the parameters of interest is included. Previous results of Kiefer and Wolfowitz (1959), Karlin and Studden (1966) and Atwood (1969) are generalized. In particular, a connection to Tchebycheff-type approximation of Rs-valued functions is found, which has been…
The Efron-Stein inequality is applied to prove that the kernel density estimate fn, with an arbitrary nonnegative kernel and an arbitrary smoothing factor, satisfies the inequality var(∫∣fn−f∣)≤4/n for all densities f. Similar inequalities are obtained for other estimates.
Asymptotic Behaviour of S-Estimates of Multivariate Location Parameters and Dispersion Matrices
It is shown under appropriate conditions that Rousseeuw's minimum volume estimator and other S-estimators of multivariate location and dispersion parameters are consistent. Under certain differentiability conditions the estimates are asymptotically normally distributed with a norming factor of n1/2.
An empirical Bayes test for testing ϑ≤ϑ0 against ϑ>ϑ0 for the uniform distribution on [0,ϑ) is discussed. The relation is shown with the estimation of a decreasing density on [0,∞) and a monotone empirical Bayes test is derived based on the least-concave majorant of the empirical distribution function. The asymptotic distribution of the Bayes risk is obtained and some Monte Carlo results are given.
Let Xi,i=1,2,…,k, be independent random variables distributed according to a one-parameter exponential family with parameter θi. Assume also that the probability density function of Xi is a Polya frequency function of order two (PF2). Consider the null hypothesis H0:θ1=θ2=⋯=θk against the alternative K: not H0. We show that any permutation invariant test of size α, whose conditional (on T=∑i=1kXi)…
In statistical decision theory, an important question is to characterize the admissible rules. In this paper, we establish complete class theorems for estimating the noncentrality parameter of noncentral chi-square and noncentral F distributions under squared error loss. Under a minor assumption, any admissible estimator must be a generalized Bayes rule. Using this result, we prove that the positive part of the UMVUE is inadmissible.
Let X1,X2,…,Xn be independent identically distributed random variables with EX12=∞ but X1 belonging to the domain of attraction of a stable law. It is known that the sample mean Xˉn appropriately normalized converges to a stable law. It is shown here that the bootstrap version of the normalized mean has a random distribution (given the sample) whose limit is also a random distribution implying that the naive bootstrap could fail in the…