To estimate the parameters of a stationary process, Whittle (1953) introduced an approximation to the Gaussian likelihood function. Although the Whittle estimate is asymptotically efficient, the small sample behavior may be poor if the spectrum of the process contains peaks. We introduce a mathematical model that covers such small sample effects. We prove that the exact maximum likelihood estimate is still optimal in this model, whereas the Whittle estimate and the…
Stationary vector ARMA processes $x(t), t = 0, \pm 1, \pm 2, \cdots$, of $n$ components are considered that are of full rank, and the situation where there are linear functions of the future $x(t), t > 0$, and the past $x(t), t \leq 0$ (more properly the present and the past) that have unit correlation. It is shown that the number of linearly independent such pairs (i.e., the number of unit canonical correlations between future and past) is the number of zeros of the…
Asymptotic properties of estimators for the confirmatory factor analysis model are discussed. The model is identified by restrictions on the elements of the factor loading matrix; the number of restrictions may exceed that required for identification. It is shown that a particular centering of the maximum likelihood estimator derived under assumed normality of observations yields an asymptotic normal distribution that is common to a wide class of distributions of the factor…
We consider an experiment which consists of $k$ treatment groups and a control group. Let the sample means $\bar{Y}_0, \bar{Y}_1, \cdots, \bar{Y}_k$ be independent normal variates with expected values $\mu_0, \mu_1, \cdots, \mu_k$ and with variances $\sigma^2/n_0, \sigma^2/n_1, \cdots, \sigma^2/n_k$. Let $w_0, w_1, \cdots, w_k$ be positive weights and let $\mu^\ast_0, \mu^\ast_1, \cdots, \mu^\ast_k$ be the weighted least squares estimators subject to the constraints $\mu_0 \leq \mu_i, i = 1, \cdots, k$…
In monotone regression procedures one utilizes only the monotonicity of the regression function. In nonparametric regression one utilizes only the assumed smoothness. The analytic and asymptotic properties of the estimator are superior in the latter case; however, monotonicity is not guaranteed. We study a hybrid procedure that produces monotone estimators with properties similar to those of nonparametric regression estimators.
The problem of testing the correctness of a nonlinear response function against unspecified general alternatives is considered. The proposed test statistic is a modification of a nonlinear analogue to the well-known linear regression lack-of-fit test and can be used with or without replication. Asymptotically valid critical points can be obtained from a central $F$-distribution. Also, when the null model is the orthogonal projection of the true model, the test statistic is…
For a regression model $y_i = \theta(x_i) + \varepsilon_i$, the unknown function $\theta$ is estimated by least squares on a subspace $\Lambda_m = \operatorname{span}\{\psi_1, \psi, \cdots, \psi_m\}$, where the basis functions $\psi_i$ are predetermined and $m$ is varied. Assuming that the design is suitably approximated by an asymptotic design measure, a general method is presented for approximating the bias and variance in a scale of Hilbertian norms natural to the…
The Tikhonov method of regularization (MOR) estimator provides a general method for estimation of a nonparametric regression parameter in an abstract linear model with discrete noisy data. An asymptotic analysis is given in which the discrete estimation problem is approximated by a continuous one. Rates of convergence are calculated in a family of norms natural to the problem. The general theory is applied to the estimation of functions from noisy evaluations of the function…
Cox's regression model has been successfully used for censored survival data. It can be adapted to model a counting process having a periodic underlying intensity. In survival analysis, the asymptotic properties, as studied by Andersen and Gill, correspond to a large number of processes running parallel over the same time interval. Here a single point process is observed over a large number of successive periods. Cox's model can easily be adapted to this situation and…
Let $X$ be a positive random variable with Lebesgue density $f_\theta(x)$, where $\theta$ is the scale parameter, and let $Y$ be a positive random variable independent of $X$. We consider two models of truncation: the LHS model, where the data consist only of those observations of $X$ for which $X > Y$; and the RHS model, where the data consist of those observations of $X$ for which $X \leq Y$. Consider the problem of estimating $\theta^s, s \neq 0$, under a normalized…
The nonparametric maximum likelihood estimators of the distribution functions of observations in the truncation model are represented as iid mean processes, with a remainder term of order $o(n^{-1/2})$ a.s.
Maximum likelihood estimation for the covariance $R$ of a zero-mean Gaussian process is considered, with no assumptions on the covariance or the "time" parameter set $T$. It is shown that the likelihood function is a.s. unbounded in general, and a sieve estimator $\hat{R}$ is constructed. The distribution of $\hat{R}$, considered as a process on $T \times T$, can be described exactly if a certain technical assumption is satisfied concerning the bivariate series expansion of $R$…
Let $\hat{H}_n(\alpha) (0 < \alpha < 1)$ denote the length of the shortest $\alpha$-fraction of the ordered sample $X_{1:n}, X_{2:n}, \cdots, X_{n:n}$, i.e., $\hat{H}_n(\alpha) = \min\{X_{k + j:n} - X_{k:n}: 1 \leq k \leq k + j \leq n; (j + 1)/n \geq \alpha\}.$ Such quantities arise in the context of robust scale estimation. Using the concept of compact derivatives of statistical functionals, the asymptotic behaviour of $\hat{H}_n(\alpha)$ as $n \rightarrow \infty$ is…
Some standard robustness concepts, developed for linear data, are found wanting in the case of directional data. We introduce a standardized bias robustness allowing for uniform robustness considerations of statistics for bounded parameter spaces. Specifically, we verify the nonrobustness in this sense of the directional mean, the directional dispersion and the maximum likelihood estimator of the concentration parameter of the von Mises-Fisher distributions.
Minimum distance estimates are studied at the $N(\theta, 1)$ model. Estimates based on a non-Hilbertian distance $\mu (\mu = \text{Kolmogorov-Smirnov, Levy, Kuiper, variation and Prohorov})$ can exhibit very large variances, or even outright inconsistency, at distributions arbitrarily close to the model in terms of $\mu$-distance. For Hilbertian distances $(\mu = \text{Cramer-von Mises, Hellinger})$ this problem does not seem to occur. Geometric motivation for these results…
The minimum distance (MD) functional defined by a distance $\mu$ is automatically robust over contamination neighborhoods defined by $\mu$. In fact, when compared to other Fisher-consistent functionals, the MD functional was no worse than twice the minimum sensitivity to $\mu$-contamination, and at least half the best possible breakdown point. In invariant settings, the MD functional has the best attainable breakdown point against $\mu$-contamination among equivariant…
In this paper we introduce $E$-ancillarity and complete $E$-sufficiency, natural extensions of the definitions of ancillarity and complete sufficiency to a space of estimating or inference functions. These are functions of both the data and the parameter. We begin either with a space of all such functions or with a subset defined to exploit special features of a model; for example, we allow restrictions to inference functions that are linear in the observations or linear in…
We consider estimation of a regression model with long-memory stationary errors. First, we estimate the regression parameters by the least-squares estimator (LSE) and, next, those describing the correlation structure of the error terms by using the residuals obtained from the LSE. Certain regularity conditions introduced to develop the asymptotic theory no longer hold in this model. In this situation we derive asymptotic properties of the preceding estimation procedure.
A mode of a probability density $f(t)$ is a value $\theta$ that maximizes $f$. The problem of estimating the location of the mode is considered here. Estimates of the mode are considered via kernel density estimates. Previous results on this problem have several serious drawbacks. Conditions on the underlying density $f$ are imposed globally (rather than locally in a neighborhood of $\theta$). Moreover, fixed bandwidth sequences are considered, resulting in an estimate of…
In this paper we consider a partial ordering that is "between" the stochastic ordering defined by Lehmann (1955) and an ordering associated with the monotone likelihood ratio property. A tail ordering deduced from it is applied to the comparison of the asymptotic efficiencies of rank tests in the two-sample problem. In particular, we show that the asymptotic relative efficiency of two rank tests preserve this tail ordering if one score function is "more convex" than the…
Though jackknifing serves well the dual purpose of bias reduction and variance estimation, the pseudovariables it generates may not generally preserve robustness for general statistical functionals. These pseudovariables are incorporated in a differentiable functional detour of jackknifing, and along with its rationality, the related asymptotic theory is studied systematically. A two-step jackknifing is considered for the variance estimation. Second-order asymptotic…
The asymptotic distribution of quadratic forms in uniform order statistics is studied under contiguous alternatives. Using minimal conditions on the sequence of forms, the limiting distribution is shown to be the convolution of a sum of weighted noncentral chi-squares and a normal variate. The results give approximate distribution theory even when no limit exists. As an example, high-order spacings statistics are shown to have trivial asymptotic power unless the order of the…
The problem of testing a given ARMA model (in which the density of the generating white noise is unspecified) against other ARMA models is considered. A distribution-free asymptotically most powerful test, based on a generalized linear serial rank statistic, is provided against contiguous ARMA alternatives with specified coefficients. In the case when the ARMA model in the alternative has unspecified coefficients, the asymptotic sufficiency (in the sense of Le Cam) of a…
Stationary vector ARMA processes x(t),t=0,±1,±2,⋯, of n components are considered that are of full rank, and the situation where there are linear functions of the future x(t),t>0, and the past x(t),t≤0 (more properly the present and the past) that have unit correlation. It is shown that the number of linearly independent such pairs (i.e., the number of unit canonical correlations between future and past) is the number of zeros of the…
We consider an experiment which consists of k treatment groups and a control group. Let the sample means Yˉ0,Yˉ1,⋯,Yˉk be independent normal variates with expected values μ0,μ1,⋯,μk and with variances σ2/n0,σ2/n1,⋯,σ2/nk. Let w0,w1,⋯,wk be positive weights and let μ0∗,μ1∗,⋯,μk∗ be the weighted least squares estimators subject to the constraints μ0≤μi,i=1,⋯,k…
The problem of testing the correctness of a nonlinear response function against unspecified general alternatives is considered. The proposed test statistic is a modification of a nonlinear analogue to the well-known linear regression lack-of-fit test and can be used with or without replication. Asymptotically valid critical points can be obtained from a central F-distribution. Also, when the null model is the orthogonal projection of the true model, the test statistic is…
For a regression model yi=θ(xi)+εi, the unknown function θ is estimated by least squares on a subspace Λm=span{ψ1,ψ,⋯,ψm}, where the basis functions ψi are predetermined and m is varied. Assuming that the design is suitably approximated by an asymptotic design measure, a general method is presented for approximating the bias and variance in a scale of Hilbertian norms natural to the…
Let X be a positive random variable with Lebesgue density fθ(x), where θ is the scale parameter, and let Y be a positive random variable independent of X. We consider two models of truncation: the LHS model, where the data consist only of those observations of X for which X>Y; and the RHS model, where the data consist of those observations of X for which X≤Y. Consider the problem of estimating θs,s=0, under a normalized…
The nonparametric maximum likelihood estimators of the distribution functions of observations in the truncation model are represented as iid mean processes, with a remainder term of order o(n−1/2) a.s.
Maximum likelihood estimation for the covariance R of a zero-mean Gaussian process is considered, with no assumptions on the covariance or the "time" parameter set T. It is shown that the likelihood function is a.s. unbounded in general, and a sieve estimator R^ is constructed. The distribution of R^, considered as a process on T×T, can be described exactly if a certain technical assumption is satisfied concerning the bivariate series expansion of R…
Let H^n(α)(0<α<1) denote the length of the shortest α-fraction of the ordered sample X1:n,X2:n,⋯,Xn:n, i.e., H^n(α)=min{Xk+j:n−Xk:n:1≤k≤k+j≤n;(j+1)/n≥α}. Such quantities arise in the context of robust scale estimation. Using the concept of compact derivatives of statistical functionals, the asymptotic behaviour of H^n(α) as n→∞ is…
Minimum distance estimates are studied at the N(θ,1) model. Estimates based on a non-Hilbertian distance μ(μ=Kolmogorov-Smirnov, Levy, Kuiper, variation and Prohorov) can exhibit very large variances, or even outright inconsistency, at distributions arbitrarily close to the model in terms of μ-distance. For Hilbertian distances (μ=Cramer-von Mises, Hellinger) this problem does not seem to occur. Geometric motivation for these results…
The minimum distance (MD) functional defined by a distance μ is automatically robust over contamination neighborhoods defined by μ. In fact, when compared to other Fisher-consistent functionals, the MD functional was no worse than twice the minimum sensitivity to μ-contamination, and at least half the best possible breakdown point. In invariant settings, the MD functional has the best attainable breakdown point against μ-contamination among equivariant…
In this paper we introduce E-ancillarity and complete E-sufficiency, natural extensions of the definitions of ancillarity and complete sufficiency to a space of estimating or inference functions. These are functions of both the data and the parameter. We begin either with a space of all such functions or with a subset defined to exploit special features of a model; for example, we allow restrictions to inference functions that are linear in the observations or linear in…
A mode of a probability density f(t) is a value θ that maximizes f. The problem of estimating the location of the mode is considered here. Estimates of the mode are considered via kernel density estimates. Previous results on this problem have several serious drawbacks. Conditions on the underlying density f are imposed globally (rather than locally in a neighborhood of θ). Moreover, fixed bandwidth sequences are considered, resulting in an estimate of…