This paper concerns robust estimation of the parameter $\theta$ which indexes a parametric model for independent non-identically distributed data. For reasonable choices of contamination neighborhood and of what is to be estimated when the parametric model does not hold, we characterize asymptotically minimax robust estimates of $\theta$. When applied to the normal regression model, the theory yields recipes for the influence curves of optimal robust regression and scale…
Maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space. For example, the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an $\operatorname{iid}$ sample; the maximum of the likelihood is not attained by any density. In this example, as in many other examples, the parameter space (positive functions with area one) is too big. But the likelihood method can often…
A method is described for constructing sequential test boundaries for which the sample size distribution is easily computed. Assuming a diffusion approximation to be adequate, a few suitably placed images with negative weights are added to a basic unit source at the origin to create absorbing boundaries with intuitively desirable properties. A simple family of closed boundaries generated in this way is compared with Ferebee's (1980) reversed parabolic boundaries and…
David C. Hamilton · Donald G. Watts · Douglas M. Bates·刊期 1982-06-01 · 卷 10(2)·The Annals of Statistics已编入正刊
非线性回归参数推断区域中的内在非线性AI 解读
Joint confidence and likelihood regions for the parameters in nonlinear regression models can be defined using the geometric concepts of sample space and solution locus. Using a quadratic approximation to the solution locus, instead of the usual linear approximation, it is shown that these inference regions correspond to ellipsoids on the tangent plane at the least squares point. Accurate approximate inference regions can be obtained by mapping these ellipsoids into the…
The differential-geometrical framework is given for analyzing statistical problems related to multi-parameter families of distributions. The dualistic structures of the exponential families and curved exponential families are elucidated from the geometrical viewpoint. The duality connected by the Legendre transformation is thus extended to include two kinds of affine connections and two kinds of curvatures. The second-order information loss is calculated for Fisher-efficient…
This paper discusses five questions concerning maximum likelihood estimation: What kind of theory is maximum likelihood? How is maximum likelihood used in practice? To what extent can this theory and practice be justified from a decision-theoretic viewpoint? What are maximum likelihood's principal virtues and defects? What improvements have been suggested by decision theory?
The integral of a second-order stochastic process $Z$ over a $d$-dimensional domain is estimated by a weighted linear combination of observations of $Z$ in a random design. The design sample points are possibly dependent random variables and are independent of the process $Z$, which may be nonstationary. Necessary and sufficient conditions are obtained for the mean squared error of a random design estimator to converge to zero as the sample size increases towards infinity.…
We consider conditions in a nonlinear regression model for the consistency of the estimator obtained by minimizing the $L_1$-norm, i.e. the sum of absolute deviations.
A general class of recursive residuals is defined by means of lower-triangular, orthonormal transformations. For these residuals, some weak invariance principles are established under appropriate regularity conditions. The theory is then incorporated in the study of robustness of some tests for change of parameters occurring at unknown time points.
In testing hypotheses involving order restrictions on a collection of parameters, distributions arise which are mixtures of standard distributions. Since tractable expressions for the mixing proportions generally do not exist even for parameter collections of moderate size, the implementation of these tests may be difficult. Stochastic upper and lower bounds are obtained for such test statistics in a variety of these kinds of problems. These bounds are also shown to be…
The maximum likelihood estimator of a nondecreasing regression function with normally distributed errors has been considered in the literature. Its asymptotic distribution at a point is related to a solution of the heat equation, and its rate of convergence to the underlying regression function is of order $n^{-1/3}$. This estimator can be modified by grouping adjacent observations and then "isotonizing" the corresponding means. It is shown that the resulting estimator has…
The problem of combining several independent Chi squared or $F$ tests is considered. The data consist of $n$ independent Chi squared or $F$ variables on which tests of the null hypothesis that all noncentrality parameters are zero are based. In each case, necessary conditions and sufficient conditions for a test to be admissible are given in terms of the monotonicity and convexity of the acceptance region. The admissibility or inadmissibility of several tests based upon the…
In a linear (or affine) functional model the principal parameter is a subspace (respectively an affine subspace) in a finite dimensional inner product space, which contains the means of $n$ multivariate normal populations, all having the same covariance matrix. A relatively simple, essentially algebraic derivation of the maximum likelihood estimates is given, when these estimates are based on single observed vectors from each of the $n$ populations and an independent…
The concept of Choquet's 2-alternating capacity is explored from the viewpoint of Le Cam's experiment theory. It is shown that there always exists a least informative binary experiment for two sets of probability measures generated by 2-alternating capacities. This result easily implies the Neyman-Pearson lemma for capacities. Moreover, its proof gives a new method of construction of minimax tests for problems in which hypotheses are generated by 2-alternating capacities. It…
Let $X_1, X_2, \cdots, X_n$ be i.i.d random variables with d.f. $F$. Suppose the $\{\hat{T}_n = \hat{T}_n(X_1, X_2, \cdots, X_n); n \geq 1\}$ are real-valued statistics and the $\{T_n(F); n \geq 1\}$ are centering functionals such that the asymptotic distribution of $n^{1/2}\{\hat{T}_n - T_n(F)\}$ is normal with mean zero. Let $H_n(x, F)$ be the exact d.f. of $n^{1/2}\{\hat{T}_n - T_n(F)\}$. The problem is to estimate $H_n(x, F)$ or functionals of $H_n(x, F)$. Under…
An asymptotic notion of robust tests is studied which is based on the requirement of equicontinuous error probabilities. If the test statistics are consistent, their robustness in Hampel's sense and robustness of the associated tests turn out to be equivalent. Uniform extensions are considered. Moreover, test breakdown points are defined. The main applications are on rank statistics: they are generally robust, under a slight condition even uniformly so; their points of final…
For a fixed probability $0 < \gamma < 1$, the "most outlying" $100(1 - \gamma){\tt\%}$ subset of the data from a location model may be located with a Grubbs outlier subset test statistic. This subset is essentially located in terms of its complement, which is the connected $100\gamma{\tt\%}$ span of the data which supports the smallest sample variance. We show that this range of the data may be characterized approximately as the $100\gamma{\tt\%}$ span such that its midpoint…
The coverage of a random sample from a multinomial population is defined to be the sum of the probabilities of the observed classes. The problem is to estimate the coverage of a random sample given only the number of classes observed exactly once, twice, etc. This problem is related to the problem of estimating the number of classes in the population. Non-parametric confidence intervals are given when the coverage is low such that a Poisson approximation holds. These…
Let $(X_1, X_2, \cdots)$ be a sequence of random variables and let the p.d.f. of $\mathbf{X}_n = (X_1, \cdots, X_n)$ be $p(\mathbf{x}_n, \theta)$, where $\theta = (\theta_1, \theta_2)$. An estimating equation rule for $\theta_1$ is a sequence of functions $g(x_1, \theta_1), g(x_1, x_2, \theta_1), \cdots$. If the random sample size $N = n$, we estimate $\theta_1$ through the estimating equation $g(\mathbf{X}_n, \theta_1) = 0$. In this paper, optimum estimation rules are…
A central limit theorem is proved for the sample covariances of a linear process. The sufficient conditions for the theorem are described by more natural ones than usual. We apply this theorem to the parameter estimation of a fitted spectral model, which does not necessarily include the true spectral density of the linear process. We also deal with estimation problems for an autoregressive signal plus white noise. A general result is given for efficiency of Newton-Raphson…
Let $\mathbf{R}$ be an infinite dimensional stationary covariance matrix, let $\mathbf{R}(k)$ and $\mathbf{W}(k)$ denote the top $k \times k$ left hand corners of $\mathbf{R}$ and $\mathbf{R}^{-1}$ respectively and let $\mathbf{\Sigma}(k)$ and $\mathbf{\Gamma}(k)$ denote the approximations for $\mathbf{R}(k)^{-1}$ suggested by Whittle (1951) and Shaman (1976) respectively. We consider quadratic forms of the type $Q(k) = \beta(k)' \mathbf{R}(k)^{-1}\alpha (k)$, when the…
This paper is concerned with the development and application of diagnostic checks for vector linear time series models. A hypothesis testing procedure based upon the score, or Lagrangean multiplier, principle is advocated and the distributions of the test statistic both under the null hypothesis and under a Pitman sequence of alternatives are discussed. Consideration of alternative models with singular sensitivity matrices when the null hypothesis is true leads to an…
A general class of models for analysis of censored survival data with covariates is considered. If $n$ individuals are observed over a time period divided into $I(n)$ intervals, it is assumed that $\lambda_j(t)$, the hazard rate function of the time to failure of the individual $j$, is constant and equal to $\lambda_{ij} > 0$ on the $i$th interval, and that the vector $\ell = \{\log \lambda_{ij}: j = 1, \ldots, n; i = 1, \ldots, I(n)\}$ lies in a linear subspace. The maximum…
This paper concerns robust estimation of the parameter θ which indexes a parametric model for independent non-identically distributed data. For reasonable choices of contamination neighborhood and of what is to be estimated when the parametric model does not hold, we characterize asymptotically minimax robust estimates of θ. When applied to the normal regression model, the theory yields recipes for the influence curves of optimal robust regression and scale…
Maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space. For example, the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an iid sample; the maximum of the likelihood is not attained by any density. In this example, as in many other examples, the parameter space (positive functions with area one) is too big. But the likelihood method can often…
The integral of a second-order stochastic process Z over a d-dimensional domain is estimated by a weighted linear combination of observations of Z in a random design. The design sample points are possibly dependent random variables and are independent of the process Z, which may be nonstationary. Necessary and sufficient conditions are obtained for the mean squared error of a random design estimator to converge to zero as the sample size increases towards infinity.…
The Consistency of Nonlinear Regression Minimizing the L1-Norm
We consider conditions in a nonlinear regression model for the consistency of the estimator obtained by minimizing the L1-norm, i.e. the sum of absolute deviations.
The maximum likelihood estimator of a nondecreasing regression function with normally distributed errors has been considered in the literature. Its asymptotic distribution at a point is related to a solution of the heat equation, and its rate of convergence to the underlying regression function is of order n−1/3. This estimator can be modified by grouping adjacent observations and then "isotonizing" the corresponding means. It is shown that the resulting estimator has…
Combining Independent Noncentral Chi Squared or F Tests
The problem of combining several independent Chi squared or F tests is considered. The data consist of n independent Chi squared or F variables on which tests of the null hypothesis that all noncentrality parameters are zero are based. In each case, necessary conditions and sufficient conditions for a test to be admissible are given in terms of the monotonicity and convexity of the acceptance region. The admissibility or inadmissibility of several tests based upon the…
In a linear (or affine) functional model the principal parameter is a subspace (respectively an affine subspace) in a finite dimensional inner product space, which contains the means of n multivariate normal populations, all having the same covariance matrix. A relatively simple, essentially algebraic derivation of the maximum likelihood estimates is given, when these estimates are based on single observed vectors from each of the n populations and an independent…
Let X1,X2,⋯,Xn be i.i.d random variables with d.f. F. Suppose the {T^n=T^n(X1,X2,⋯,Xn);n≥1} are real-valued statistics and the {Tn(F);n≥1} are centering functionals such that the asymptotic distribution of n1/2{T^n−Tn(F)} is normal with mean zero. Let Hn(x,F) be the exact d.f. of n1/2{T^n−Tn(F)}. The problem is to estimate Hn(x,F) or functionals of Hn(x,F). Under…
For a fixed probability 0<γ<1, the "most outlying" 100(1−γ)% subset of the data from a location model may be located with a Grubbs outlier subset test statistic. This subset is essentially located in terms of its complement, which is the connected 100γ% span of the data which supports the smallest sample variance. We show that this range of the data may be characterized approximately as the 100γ% span such that its midpoint…
Let (X1,X2,⋯) be a sequence of random variables and let the p.d.f. of Xn=(X1,⋯,Xn) be p(xn,θ), where θ=(θ1,θ2). An estimating equation rule for θ1 is a sequence of functions g(x1,θ1),g(x1,x2,θ1),⋯. If the random sample size N=n, we estimate θ1 through the estimating equation g(Xn,θ1)=0. In this paper, optimum estimation rules are…
Let R be an infinite dimensional stationary covariance matrix, let R(k) and W(k) denote the top k×k left hand corners of R and R−1 respectively and let Σ(k) and Γ(k) denote the approximations for R(k)−1 suggested by Whittle (1951) and Shaman (1976) respectively. We consider quadratic forms of the type Q(k)=β(k)′R(k)−1α(k), when the…
A general class of models for analysis of censored survival data with covariates is considered. If n individuals are observed over a time period divided into I(n) intervals, it is assumed that λj(t), the hazard rate function of the time to failure of the individual j, is constant and equal to λij>0 on the ith interval, and that the vector ℓ={logλij:j=1,…,n;i=1,…,I(n)} lies in a linear subspace. The maximum…