Exponential response models are a generalization of logit models for quantal responses and of regression models for normal data. In an exponential response model, $\{F(\theta): \theta \in \Theta\}$ is an exponential family of distributions with natural parameter $\theta$ and natural parameter space $\Theta \subset V$, where $V$ is a finite-dimensional vector space. A finite number of independent observations $S_i, i \in I$, are given, where for $i \in I, S_i$ has…
For the one-sample $t$-test a new form of the exact distribution of the test statistic $t^2$ is obtained when sampling from a distribution which is a mixture of two normal distributions. A numerical example is provided to show that the size of the test can differ greatly when sampling from distributions having the same skewness and kurtosis. Contours of equal size are plotted for a particular case in a certain cross section of the parameter space.
This paper is concerned with the minimum variance estimation of a time-dependent population mean, assuming that one is restricted to the case of linear unbiased estimators. A number of results are given for a new rotation sampling model (RSM), in which unequal sample sizes are used on each occasion. Also results corresponding to the special case of sampling with a fixed sample size on all the occasions are derived. Finally the optimum structure of the suggested model is…
Let $X_1, \cdots, X_n$ be i.i.d. $F$, and estimate the median of $F$ by the median $T_\beta$ of $\beta X_i + (1 - \beta)X_j, i \neq j$, where $\beta$ is a fixed positive constant. Then $T_\beta$ is the solution of a $U$-statistic equation from which its asymptotic normality is readily derived. The asymptotic relative efficiency of $T_\beta$ is computed for a few cdfs $F$ and seen to be reasonably high for unintuitive choices such as $\beta = .9, \beta = 2$, and also to be…
For estimating the mean in the one parameter exponential family with quadratic loss, Karlin (1958) gave sufficient conditions for admissibility of estimators of the form $aX$. Later, Ping (1964) and Gupta (1966) gave sufficient conditions for admissibility of estimators of the form $aX + b$ for the same problem. Zidek (1970) gave sufficient conditions for the admissibility of $X$ for estimating an arbitrary piecewise continuous function of the parameter, say $\gamma(\theta)$,…
J. Berger · M. E. Bock · L. D. Brown · G. Casella · L. Gleser·刊期 1977-07-01 · 卷 5(4)·The Annals of Statistics已编入正刊
未知协方差与任意二次损失下正态均值向量的极小极大估计AI 解读
Let $X$ be an observation from a $p$-variate normal distribution $(p \geqq 3)$ with mean vector $\theta$ and unknown positive definite covariance matrix $\not\Sigma$. It is desired to estimate $\theta$ under the quadratic loss $L(\delta, \theta, \not\Sigma) = (\delta - \theta)^tQ(\delta - \theta)/\operatorname{tr} (Q\not\Sigma)$, where $Q$ is a known positive definite matrix. Estimators of the following form are considered: $\delta^c(X, W) = (I - c\alpha Q^{-1}W^{-1}/(X^tW^{-1}X))X,$…
We show that in the mixed model of the analysis of variance, there is a sequence of roots of the likelihood equations which is consistent, asymptotically normal, and efficient in the sense of attaining the Cramer-Rao lower bound for the covariance matrix. These results follow directly by an application of a general result of Weiss (1971, 1973) concerning maximum likelihood estimates. This problem differs from standard problems in that we do not have independent, identically…
Let $T \subseteqq I$ be sets of real numbers. Let $\{Y(t): t \in I\}$ be a real time series whose mean is an unknown element of a known class of functions on $I$ and whose covariance kernel $k$ is assumed known. For each fixed $s \in I, Y(s)$ is predicted by a minimum mean square error unbiased linear predictor $\hat{Y}(s)$ based on $\{Y(t): t \in T\}$. If $\hat{y}(s)$ is the evaluation of $\hat{Y}(s)$ given a set of observations $\{Y(t) = g(t): t \in T\}$, then the function…
For the autoregressive-moving average time series model, the normal theory procedure for setting confidence intervals for the error variance is not robust against nonnormality. This paper proposes three asymptotically robust techniques: they are a "standard-error" procedure, an analog of Box's simple data splitting technique, and the jackknife procedure. The large sample distribution of each of these techniques is derived.
This paper proves the asymptotic linearity in the regression parameter of a class of linear rank statistics when errors in the regression model are strictly stationary and strongly mixing. Besides this, several other weak convergence results are proved which yield the asymptotic normality of $L$ and $M$ estimators of the regression parameter under the above dependent structure. All these results are useful in studying the effect of the above dependence on the asymptotic…
A new approach to the asymptotic normality of simple linear rank statistics for the regression case studied earlier by Hajek (1968) is provided along with the estimation of the remainder term in the approximation to normality.
If $X_1, \cdots, X_n$ is a random sample from $F(x - \theta)$, where $F$ is an unknown member of a specified class $\mathscr{F}$ of approximately normal symmetric distributions, then an $M$-estimator of the unknown location parameter $\theta$ is obtained by solving the equation $\sum^n_{i=1} \psi(X_i - \hat{\theta}_n) = 0$ for $\hat{\theta}_n$. A suitable measure of the robustness of the $M$-estimator is $\sup \{V(\psi, F): F \in \mathscr{F}\}$, where $V(\psi, F) = \int \psi^2 dF/(\int \psi' dF)^2$…
Convergence rates for distributions of simple linear rank statistics are investigated. Both the null hypothesis and near alternatives are considered. The method of proof consists in approximating the characteristic function of the statistic by that of a sum of independent random variables and then applying standard tools.
Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$-valued and $Y$ is $\mathbb{R}^{d'}$-valued. Given a random sample $(X_1, Y_1), \cdots, (X_n, Y_n)$ from the distribution of $(X, Y)$, the conditional distribution $P^Y(\bullet \mid X)$ of $Y$ given $X$ can be estimated nonparametrically by $\hat{P}_n^Y(A \mid X) = \sum^n_1 W_{ni}(X)I_A(Y_i)$, where the weight function $W_n$ is of the form $W_{ni}(X) = W_{ni}(X, X_1, \cdots, X_n), 1 \leqq i \leqq n$.…
Let $y_t$ be an order $p$ autoregressive process of the form $y_t + \sum^p_{s=1} \beta_s y_{t-s} = u_t$, where the $u_t$'s are i.i.d. variables with a symmetric distribution $F$ such that $E \log^+ |u_t| < \infty$. For the Yule-Walker version $\beta_T^\ast$ of the least-squares estimate of $\beta = (\beta_1,\cdots, \beta_p)$, it is shown that $T^\frac{1}{2}(\beta_T^\ast - \beta)$ is bounded in probability.
An estimator for the spectral density of a stationary time sequence is introduced. The form of the estimator is motivated by the analogy with the summability theory of Fourier series. A theorem relating to rate of consistency is proved. The effect of a mean correction is considered.
The rate at which the mean integrated square error decreases as sample size increases is evaluated for general $L^1$ kernel estimates and for the Fourier integral estimate for a probability density. The rates are compared to that of the minimum M.I.S.E.; the Fourier integral estimate is found to be asymptotically optimal.
Consider a regular $p$-dimensional exponential family such that either the distributions are concentrated on a lattice or they have a component whose $k$-fold convolution has a bounded density with respect to Lebesgue measure. Then, if a parametric function has an unbiased estimator, the minimum variance unbiased estimators are asymptotically equivalent to the maximum likelihood estimators; and, hence, are asymptotically efficient. Examples are given to show that a condition…
In the context of the two-action, linear loss, exponential family multiple decision problem, Van Houwelingen (1973), (1976) has shown that faster rates of convergence are deducible for monotone empirical Bayes procedures than result from application of the bound established for general empirical Bayes procedures by Johns and Van Ryzin (1972). This note generalizes a (1973) Van Houwelingen bound to arbitrary $k$-action, monotone loss, MLR family multiple decision problems. An…
An $n$-variate distribution function is said to be positive dependent by mixture (PDM) if it is a mixture of independent $n$-variate distributions with equal marginals. PDM distributions arise in various contexts of reliability and other areas of statistics. We give a necessary and sufficient condition, by means of independent random variables, for an $n$-variate distribution function to be PDM. The distributions and the expectations of the order statistics of PDM and of…
A general law of the iterated logarithm for linear combinations of order statistics is proved. The key tools are (1) iterated logarithm convergence of the uniform empirical process $U_n$ in $\rho_q$-metrics due to B. R. James and (2) almost sure "nearly linear" bounds for the empirical distribution function. A law of the iterated logarithm for the quantile process is also established.
A strengthened version of the Glivenko-Cantelli theorem for the uniform empirical distribution function is proved. The strengthened Glivenko-Cantelli theorem is used to establish strong laws of large numbers for linear functions of order statistics.
Let $\hat{\mathbf{\Delta}}_M$ be an $M$-estimator (maximum-likelihood type estimator) and $\hat{\mathbf{\Delta}}_R$ be an $R$-estimator (rank estimator) of the parameter $\mathbf{\Delta} = (\Delta_1,\cdots, \Delta_p)$ in the linear regression model $X_{Ni} = \sum^p_{j=1} \Delta_jc_{ji} + e_i, i = 1,\cdots, N$. The asymptotic distribution of $\hat\mathbf{\Delta}_M - \hat\mathbf{\Delta}_R$ is derived for $p$ fixed and $N \rightarrow \infty,$ under some assumptions on the…
This paper defines and studies for independent identically distributed observations a new parametric estimation procedure which is asymptotically efficient under a specified regular parametric family of densities and is minimax robust in a small Hellinger metric neighborhood of the given family. Associated with the estimator is a goodness-of-fit statistic which assesses the adequacy of the chosen parametric model. The fitting of a normal location-scale model by the new…
Exponential response models are a generalization of logit models for quantal responses and of regression models for normal data. In an exponential response model, {F(θ):θ∈Θ} is an exponential family of distributions with natural parameter θ and natural parameter space Θ⊂V, where V is a finite-dimensional vector space. A finite number of independent observations Si,i∈I, are given, where for i∈I,Si has…
One-Sample t-Test When Sampling from a Mixture of Normal Distributions
For the one-sample t-test a new form of the exact distribution of the test statistic t2 is obtained when sampling from a distribution which is a mixture of two normal distributions. A numerical example is provided to show that the size of the test can differ greatly when sampling from distributions having the same skewness and kurtosis. Contours of equal size are plotted for a particular case in a certain cross section of the parameter space.
A Location Estimator Based on a U-Statistic
Let X1,⋯,Xn be i.i.d. F, and estimate the median of F by the median Tβ of βXi+(1−β)Xj,i=j, where β is a fixed positive constant. Then Tβ is the solution of a U-statistic equation from which its asymptotic normality is readily derived. The asymptotic relative efficiency of Tβ is computed for a few cdfs F and seen to be reasonably high for unintuitive choices such as β=.9,β=2, and also to be…
For estimating the mean in the one parameter exponential family with quadratic loss, Karlin (1958) gave sufficient conditions for admissibility of estimators of the form aX. Later, Ping (1964) and Gupta (1966) gave sufficient conditions for admissibility of estimators of the form aX+b for the same problem. Zidek (1970) gave sufficient conditions for the admissibility of X for estimating an arbitrary piecewise continuous function of the parameter, say γ(θ),…
Let X be an observation from a p-variate normal distribution (p≧3) with mean vector θ and unknown positive definite covariance matrix Σ. It is desired to estimate θ under the quadratic loss L(δ,θ,Σ)=(δ−θ)tQ(δ−θ)/tr(QΣ), where Q is a known positive definite matrix. Estimators of the following form are considered: δc(X,W)=(I−cαQ−1W−1/(XtW−1X))X,…
Let T⫅I be sets of real numbers. Let {Y(t):t∈I} be a real time series whose mean is an unknown element of a known class of functions on I and whose covariance kernel k is assumed known. For each fixed s∈I,Y(s) is predicted by a minimum mean square error unbiased linear predictor Y^(s) based on {Y(t):t∈T}. If y^(s) is the evaluation of Y^(s) given a set of observations {Y(t)=g(t):t∈T}, then the function…
This paper proves the asymptotic linearity in the regression parameter of a class of linear rank statistics when errors in the regression model are strictly stationary and strongly mixing. Besides this, several other weak convergence results are proved which yield the asymptotic normality of L and M estimators of the regression parameter under the above dependent structure. All these results are useful in studying the effect of the above dependence on the asymptotic…
Upper Bounds on Asymptotic Variances of M-Estimators of Location
If X1,⋯,Xn is a random sample from F(x−θ), where F is an unknown member of a specified class F of approximately normal symmetric distributions, then an M-estimator of the unknown location parameter θ is obtained by solving the equation ∑i=1nψ(Xi−θ^n)=0 for θ^n. A suitable measure of the robustness of the M-estimator is sup{V(ψ,F):F∈F}, where V(ψ,F)=∫ψ2dF/(∫ψ′dF)2…
Let (X,Y) be a pair of random variables such that X is Rd-valued and Y is Rd′-valued. Given a random sample (X1,Y1),⋯,(Xn,Yn) from the distribution of (X,Y), the conditional distribution PY(∙∣X) of Y given X can be estimated nonparametrically by P^nY(A∣X)=∑1nWni(X)IA(Yi), where the weight function Wn is of the form Wni(X)=Wni(X,X1,⋯,Xn),1≦i≦n.…
Let yt be an order p autoregressive process of the form yt+∑s=1pβsyt−s=ut, where the ut's are i.i.d. variables with a symmetric distribution F such that Elog+∣ut∣<∞. For the Yule-Walker version βT∗ of the least-squares estimate of β=(β1,⋯,βp), it is shown that T21(βT∗−β) is bounded in probability.
The rate at which the mean integrated square error decreases as sample size increases is evaluated for general L1 kernel estimates and for the Fourier integral estimate for a probability density. The rates are compared to that of the minimum M.I.S.E.; the Fourier integral estimate is found to be asymptotically optimal.
Consider a regular p-dimensional exponential family such that either the distributions are concentrated on a lattice or they have a component whose k-fold convolution has a bounded density with respect to Lebesgue measure. Then, if a parametric function has an unbiased estimator, the minimum variance unbiased estimators are asymptotically equivalent to the maximum likelihood estimators; and, hence, are asymptotically efficient. Examples are given to show that a condition…
Improved Rates in the Empirical Bayes Monotone Multiple Decision Problem MLR Family
In the context of the two-action, linear loss, exponential family multiple decision problem, Van Houwelingen (1973), (1976) has shown that faster rates of convergence are deducible for monotone empirical Bayes procedures than result from application of the bound established for general empirical Bayes procedures by Johns and Van Ryzin (1972). This note generalizes a (1973) Van Houwelingen bound to arbitrary k-action, monotone loss, MLR family multiple decision problems. An…
An n-variate distribution function is said to be positive dependent by mixture (PDM) if it is a mixture of independent n-variate distributions with equal marginals. PDM distributions arise in various contexts of reliability and other areas of statistics. We give a necessary and sufficient condition, by means of independent random variables, for an n-variate distribution function to be PDM. The distributions and the expectations of the order statistics of PDM and of…
A general law of the iterated logarithm for linear combinations of order statistics is proved. The key tools are (1) iterated logarithm convergence of the uniform empirical process Un in ρq-metrics due to B. R. James and (2) almost sure "nearly linear" bounds for the empirical distribution function. A law of the iterated logarithm for the quantile process is also established.
Asymptotic Relations of M-Estimates and R-Estimates in Linear Regression Model
Let Δ^M be an M-estimator (maximum-likelihood type estimator) and Δ^R be an R-estimator (rank estimator) of the parameter Δ=(Δ1,⋯,Δp) in the linear regression model XNi=∑j=1pΔjcji+ei,i=1,⋯,N. The asymptotic distribution of $\hat\mathbf{\Delta}_M - \hat\mathbf{\Delta}_R$ is derived for p fixed and N→∞, under some assumptions on the…