Standard large-sample maximum likelihood and Bayesian inference, based on limiting multivariate normal distributions, may be dubious when applied with small or moderate sample sizes. We define and discuss several measures of nonnormality of MLE and posterior distributions that may be used as diagnostics and can indicate whether reparameterization will be effective in improving inferences. We begin by showing how the nonlinearity measures introduced by Beale and Bates and…
Toby Mitchell · Jerome Sacks · Donald Ylvisaker·刊期 1994-06-01 · 卷 22(2)·The Annals of Statistics已编入正刊
非参数响应面设计的渐近贝叶斯准则AI 解读
This paper deals with Bayesian design for response surface prediction when the prior may be finite or infinite dimensional, the design space arbitrary. In order that the resulting problems be manageable, we resort to asymptotic versions of D-, G- and A-optimality. Here the asymptotics stem from allowing the error variance to be large. The problems thus elicited have strong game-like characteristics. Examples of theoretical solutions are brought forward, especially when the…
Probability inequalities governing the approximation of the basic martingale, the rescaled difference between a counting process and its compensator, are derived. Applications to the random censoring model and to the field of goodness-of-fit tests are given. The inequalities are compared to inequalities derived earlier for the random censoring model using an empirical process approach.
The local median regression method has long been known as a robustified alternative to methods such as local mean regression. Yet, its optimal statistical properties are largely unknown. In this paper, we show via decision-theoretic arguments that a local weighted median estimator is the best least absolute deviation estimator in an asymptotic minimax sense, under $L_1$-loss. We also study asymptotic efficiency of the local median estimator in the class of all possible…
The monotone class rank-test-based estimating equations for regression models with right censored data is considered. We introduce an estimator which is a solution of a monotone estimating equation that is an extension of the Gehan test. The estimator is easy to derive, $\sqrt n$-consistent and asymptotically normal under minimal conditions. All monotone estimating equations are characterized, and a simulation study, which shows that our suggested procedure performs well, is…
In this paper, we develop a general theory for the convergence rate of sieve estimates, maximum likelihood estimates (MLE's) and related estimates obtained by optimizing certain empirical criteria in general parameter spaces. In many cases, especially when the parameter space is infinite dimensional, maximization over the whole parameter space is undesirable. In such cases, one has to perform maximization over an approximating space (sieve) of the original parameter space…
This paper establishes the uniform closeness of a randomly weighted residual empirical process to its natural estimator via weak convergence techniques. The weights need not be independent, bounded or even square integrable. This result is used to yield the asymptotic uniform linearity of a class of rank statistics in $p$th-order autoregression models. The latter result, in turn, yields the asymptotic distributions of a class of robust and Jaeckel-type rank estimators. The…
We study problems of semiparametric statistical inference connected with long-memory covariance stationary time series, having spectrum which varies regularly at the origin: There is an unknown self-similarity parameter, but elsewhere the spectrum satisfies no parametric or smoothness conditions, it need not be in $L_p$, for any $p > 1$, and in some circumstances the slowly varying factor can be of unknown form. The basic statistic of interest is the discretely averaged…
A new approach to the problem of identifying a nonlinear time series model is considered. We argue that cumulative lagged conditional mean and variance functions are the appropriate "signatures" of a nonlinear time series for the purpose of model identification, being analogous to cumulative distribution functions or cumulative hazard functions in iid models. We introduce estimators of the cumulative lagged conditional mean and variance functions and study their asymptotic…
For a random sample of size $n$ from an absolutely continuous bivariate population $(X, Y)$, let $X_{i:n}$ denote the $i$th order statistic of the $X$ sample values. The $Y$-value associated with $X_{i:n}$ is denoted by $Y_{\lbrack i:n\rbrack}$ and is called the concomitant of the $i$th order statistic. For $1 \leq k \leq n$, let $V_{k,n} = \max(Y_{\lbrack n - k + 1: n\rbrack},\ldots,Y_{\lbrack n: n\rbrack})$. In this paper, we discuss the finite-sample and the asymptotic…
We introduce a general class of conditional $U$-statistics and present sufficient conditions for their universal consistency in $r$th mean. It is shown that under mild assumptions on the smoothing parameters, window and $k_n$-nearest neighbor estimators are universally consistent. An application to a new nonparametric discrimination problem is also included.
This paper extends the study of Wishart and multivariate beta distributions to the singular case, where the rank is below the dimensionality. The usual conjugacy is extended to this case. A volume element on the space of positive semidefinite $m \times m$ matrices of rank $n < m$ is introduced and some transformation properties established. The density function is found for all rank-$n$ Wishart distributions as well as the rank-1 multivariate beta distribution. To do that,…
A class of exponential transformation models is defined on symmetric cones $\Omega$ with the group of automorphisms on $\Omega$ as the acting group. We show that these models are reproductive and the exponent of their joint distribution for a given sample of size $q$ can be split into $q$ independent components, introducing one sample point at a time. The automorphism group can be factorized into the group of positive dilation and another group. Accordingly, the symmetric…
We consider the Hotelling $T^2$ statistic for an arbitrary $d$-dimensional sample. If the sampling is not too deterministic or inhomogeneous, then under the zero-means hypothesis the limiting distribution for $T^2$ is $\chi^2_d$. It is shown that a test for the orthant symmetry condition introduced by Efron can be constructed which does not differ essentially from the one based on $\chi^2_d$ and at the same time is applicable not only to large random homogeneous samples but…
The fundamental result on the rate of convergence of the EM algorithm has proven to be theoretically valuable and practically useful. Here, this result is generalized to the ECM algorithm, a more flexible and applicable iterative algorithm proposed recently by Meng and Rubin. Results on the rate of convergence of variations of ECM are also presented. An example is given to show that intuitions accurate for complete-data iterative algorithms may not be trustworthy in the…
For some time, so-called empirical likelihoods have been used heuristically for purposes of nonparametric estimation. Owen showed that empirical likelihood ratio statistics for various parameters $\theta(F)$ of an unknown distribution $F$ have limiting chi-square distributions and may be used to obtain tests or confidence intervals in a way that is completely analogous to that used with parametric likelihoods. Our objective in this paper is twofold: first, to link estimating…
In the point and interval estimation of the variance of a normal distribution with an unknown mean, the best affine equivariant estimators are dominated by Stein's truncated and Brewster and Zidek's smooth procedures, which are separately derived. This paper gives a unified approach to this problem by using a simple definite integral and provides a class of improved procedures in both point and interval estimation of powers of the scale parameter of normal, lognormal,…
Mallows has conjectured that among distributions which are Gaussian but for occasional contamination by additive noise, the one having least Fisher information has (two-sided) geometric contamination. A very similar problem arises in estimation of a nonnegative vector parameter in Gaussian white noise when it is known also that most [i.e., $(1 - \varepsilon)$] components are zero. We provide a partial asymptotic expansion of the minimax risk as $\varepsilon \rightarrow 0$.…
New formulas are given for the minimax linear risk in estimating a linear functional of an unknown object from indirect data contaminated with random Gaussian noise. The formulas cover a variety of loss functions and do not require the symmetry of the convex a priori class. It is shown that affine minimax rules are within a few percent of minimax even among nonlinear rules, for a variety of loss functions. It is also shown that difficulty of estimation is measured by the…
Chen and Shiau showed that a two-stage spline smoothing method and the partial regression method lead to efficient estimators for the parametric component of a partially linear model when the smoothing parameter is a deterministic sequence tending to zero at an appropriate rate. This paper is concerned with the large-sample behavior of these estimators when the smoothing parameter is chosen by the generalized cross validation (GCV) method or Mallows' $C_L$. Under mild…
A modification of kernel density estimation is proposed. The first step is ordinary kernel estimation of the density and its cdf. In the second step the data are transformed, using this estimated cdf, to an approximate uniform (or normal or other target) distribution. The density and cdf of the transformed data are then estimated by the kernel method and, by change of variable, converted to new estimates of the density and the cdf of the original data. This process is…
Let $X_1, \ldots, X_M, Y_1,\ldots, Y_N$ be random variables, and set $\mathbf{X} = (X_1, \ldots, X_M)$ and $\mathbf{Y} = (Y_1, \ldots, Y_N)$. Let $\varphi$ be the regression or logistic or Poisson regression function of $\mathbf{Y}$ on $\mathbf{X}(N = 1)$ or the logarithm of the density function of $\mathbf{Y}$ or the conditional density function of $\mathbf{Y}$ on $\mathbf{X}$. Consider the approximation $\varphi^\ast$ to $\varphi$ having a suitably defined form involving a…
This article deals with the construction of confidence intervals when the components of the location parameter $\mu$ of the random variable $\mathbf{X}$, which is elliptically symmetrically distributed, are subject to order restrictions. Several domination results are proved by studying the derivative of the coverage probability of the confidence intervals centered at the improved point estimators. Consequently, we strengthen the previously known results regarding the simple…
Hajek established a local asymptotic minimax risk bound for appropriate symmetric loss functions and also gave a necessary condition for the risk of an estimator to attain the lower bound. We extend these results to the case of asymmetric loss functions. The asymmetry brings about the shift of location of the loss functions. Besides, the optimal estimator that attains the bound is shown to have asymptotic normal distribution with asymptotic bias.
The local median regression method has long been known as a robustified alternative to methods such as local mean regression. Yet, its optimal statistical properties are largely unknown. In this paper, we show via decision-theoretic arguments that a local weighted median estimator is the best least absolute deviation estimator in an asymptotic minimax sense, under L1-loss. We also study asymptotic efficiency of the local median estimator in the class of all possible…
The monotone class rank-test-based estimating equations for regression models with right censored data is considered. We introduce an estimator which is a solution of a monotone estimating equation that is an extension of the Gehan test. The estimator is easy to derive, n-consistent and asymptotically normal under minimal conditions. All monotone estimating equations are characterized, and a simulation study, which shows that our suggested procedure performs well, is…
This paper establishes the uniform closeness of a randomly weighted residual empirical process to its natural estimator via weak convergence techniques. The weights need not be independent, bounded or even square integrable. This result is used to yield the asymptotic uniform linearity of a class of rank statistics in pth-order autoregression models. The latter result, in turn, yields the asymptotic distributions of a class of robust and Jaeckel-type rank estimators. The…
We study problems of semiparametric statistical inference connected with long-memory covariance stationary time series, having spectrum which varies regularly at the origin: There is an unknown self-similarity parameter, but elsewhere the spectrum satisfies no parametric or smoothness conditions, it need not be in Lp, for any p>1, and in some circumstances the slowly varying factor can be of unknown form. The basic statistic of interest is the discretely averaged…
For a random sample of size n from an absolutely continuous bivariate population (X,Y), let Xi:n denote the ith order statistic of the X sample values. The Y-value associated with Xi:n is denoted by Y[i:n] and is called the concomitant of the ith order statistic. For 1≤k≤n, let Vk,n=max(Y[n−k+1:n],…,Y[n:n]). In this paper, we discuss the finite-sample and the asymptotic…
Universally Consistent Conditional U-Statistics
We introduce a general class of conditional U-statistics and present sufficient conditions for their universal consistency in rth mean. It is shown that under mild assumptions on the smoothing parameters, window and kn-nearest neighbor estimators are universally consistent. An application to a new nonparametric discrimination problem is also included.
This paper extends the study of Wishart and multivariate beta distributions to the singular case, where the rank is below the dimensionality. The usual conjugacy is extended to this case. A volume element on the space of positive semidefinite m×m matrices of rank n<m is introduced and some transformation properties established. The density function is found for all rank-n Wishart distributions as well as the rank-1 multivariate beta distribution. To do that,…
A class of exponential transformation models is defined on symmetric cones Ω with the group of automorphisms on Ω as the acting group. We show that these models are reproductive and the exponent of their joint distribution for a given sample of size q can be split into q independent components, introducing one sample point at a time. The automorphism group can be factorized into the group of positive dilation and another group. Accordingly, the symmetric…
Extremal Probabilistic Problems and Hotelling's T2 Test Under a Symmetry Condition
We consider the Hotelling T2 statistic for an arbitrary d-dimensional sample. If the sampling is not too deterministic or inhomogeneous, then under the zero-means hypothesis the limiting distribution for T2 is χd2. It is shown that a test for the orthant symmetry condition introduced by Efron can be constructed which does not differ essentially from the one based on χd2 and at the same time is applicable not only to large random homogeneous samples but…
For some time, so-called empirical likelihoods have been used heuristically for purposes of nonparametric estimation. Owen showed that empirical likelihood ratio statistics for various parameters θ(F) of an unknown distribution F have limiting chi-square distributions and may be used to obtain tests or confidence intervals in a way that is completely analogous to that used with parametric likelihoods. Our objective in this paper is twofold: first, to link estimating…
Mallows has conjectured that among distributions which are Gaussian but for occasional contamination by additive noise, the one having least Fisher information has (two-sided) geometric contamination. A very similar problem arises in estimation of a nonnegative vector parameter in Gaussian white noise when it is known also that most [i.e., (1−ε)] components are zero. We provide a partial asymptotic expansion of the minimax risk as ε→0.…
Chen and Shiau showed that a two-stage spline smoothing method and the partial regression method lead to efficient estimators for the parametric component of a partially linear model when the smoothing parameter is a deterministic sequence tending to zero at an appropriate rate. This paper is concerned with the large-sample behavior of these estimators when the smoothing parameter is chosen by the generalized cross validation (GCV) method or Mallows' CL. Under mild…
Let X1,…,XM,Y1,…,YN be random variables, and set X=(X1,…,XM) and Y=(Y1,…,YN). Let φ be the regression or logistic or Poisson regression function of Y on X(N=1) or the logarithm of the density function of Y or the conditional density function of Y on X. Consider the approximation φ∗ to φ having a suitably defined form involving a…
This article deals with the construction of confidence intervals when the components of the location parameter μ of the random variable X, which is elliptically symmetrically distributed, are subject to order restrictions. Several domination results are proved by studying the derivative of the coverage probability of the confidence intervals centered at the improved point estimators. Consequently, we strengthen the previously known results regarding the simple…