Measures of location differentiable at every density in the Hellinger metric are constructed in this paper. Differentiability entitles these location functionals to the label "robust," even though their influence curves need not be bounded and continuous. The latter properties are, in fact, associated with functionals differentiable in the Prokhorov metric. A Hellinger metric concept of minimax robustness of a location measure at a density shape $f$ is developed.…
Consider a sample from a multiple time series that is stationary and Gaussian. A test is presented for independence among the multivariate observations that comprise this sample. The test is a generalization of the Kolmogorov-Smirnov test for serial correlation in a single time series. In the test, pairs of spectral-matrix estimates are compared using the largest-root statistic. The comparisons, which are tested simultaneously, are between estimates obtained from upper and…
It is shown that the limiting distribution of any regular estimator of a continuous cdf on [0, 1] can be represented as a convolution of the Brownian bridge process with another distribution on $C\lbrack 0, 1\rbrack$. The result is related to Hajek's representation for limiting distributions of regular parametric estimators.
Based on a random sample from a univariate distribution with density $f$, this note exhibits a class of kernel estimators of the $p$th order derivative $f^{(p)}$ of $f, p \geqq 0$ fixed. These estimators improve some known estimators of $f^{(p)}$ by weakening the conditions, sharpening the rates of convergence, or both for the properties of strong consistency, asymptotic unbiasedness and mean square consistency, each uniform on the real line.
It is desired to estimate the difference between the means of two independent normal distributions as accurately as possible and in a sequential manner when the total number of observations is fixed. The problem is posed in a Bayesian framework with conjugate prior distributions and squared error loss function. It is shown that the optimal sequential design depends on the ratio of the posterior variances of the two means. There exist constants (dependent on the prior…
The empirical distribution function $P_n$ converges with probability 1 to a true distribution $P$ in $R^k$, uniformly over measurable convex sets, if and only if $P$ is a countable mixture of distributions, each of which is carried by a flat and gives zero probability to the relative boundaries of convex sets included in the flat.
The sequential estimation of $p$, the probability of success in a sequence of Bernoulli trials, is considered for the case where loss is taken to be symmetrized relative squared error of estimation, plus a fixed cost $c$ per observation. Using $s_n/n$ as a terminal estimator of $p$, where $s_n$ is the number of successes in $n$ trials, a heuristic rule is derived and shown to perform well for any fixed $0 < p < 1$ as $c \rightarrow 0$. However for any fixed $c > 0$, this…
Material in Chapter VI of Hajek and Sidak's book is extended to a sequential analysis setting: conditions are given under which a sequence of log-likelihood-ratio processes (log-likelihood-ratios for sequential sampling, represented as jump processes in continuous time) converges weakly to a Wiener process with drift, the drift parameter depending on which hypothesis, in a suitable neighborhood of a null hypothesis, prevails. Conditions for convergence of other "test…
The stopping time $N$ of a sequential test based on an i.i.d. sequence $X, X_1, X_2, \cdots$ with common distribution $P$ is defined as the first integer $n \geqq 1$ such that $l_1 < L_n < l_2$ is violated, where $L_n$ is a statistic depending only on $X_1, \cdots, X_n$. If for all $l_1, l_2$ there exist constants $c > 0, \rho < 1$ such that $P(N > n) < c\rho^n, n = 1, 2, \cdots,$ then $N$ is called exponentially bounded under $P$. In the contrary case $P$ is termed…
Let $\mathbf{a}$ and $\mathbf{b}$ be $r$-dimensional real vectors. It is shown that if $\mathbf{a}$ majorizes $\mathbf{b}$, then $E(\Pi^r_{j = 1} X_j^a j) \geqq E(\Pi^r_{j = 1} X_j^b j)$ holds for nonnegative random variables $X_1, \cdots, X_r$ whose joint pdf is permutation symmetric. If in addition the components of $\mathbf{a, b}$ are nonnegative integers, then for every Borel-measurable set $A$, $\Pi^r_{j = 1} P\lbrack\cap^{a_j}_{i = 1} \{Z_i \in A\}\rbrack \geqq \Pi^r_{j = 1} P\lbrack\cap^{b_j}_{i = 1} \{Z_i \in A\}\rbrack$…
In establishing weak convergence of von Mises' differentiable statistical functions to a normal distribution usually square integrability conditions with respect to the underlying kernel function are assumed. It is shown that these conditions can be weakened by assuming integrability of the von Mises' functional itself. In addition it is pointed out that in nontrivial cases the conditions of square integrability of the kernel do not hold whereas weak convergence of the von…
Sequences of tests with error $\exp(-nA)$ of the first type are investigated. It is shown that the error of the second type of such a sequence of tests is bounded by $\exp(- nB)$ where $B$ is determined by the Kullback-Leibler information distance of the hypotheses tested. The information distance between the empirical measure and the null-hypothesis on a finite partition of the sample space is proposed to use as a test statistic. A sufficient condition is given which…
A Berry-Esseen bound is obtained for trimmed linear combinations of order statistics. These linear combinations are written as the sum of a linear and a quadratic combination of independent exponentially distributed random variables plus a remainder term. The remainder term is shown to be of negligible order and Fourier methods are then employed to handle the linear and quadratic terms. The main theorem is also given in a version that more easily lends itself to applications.
A Robbins-Monro type procedure for estimating the zero of a regression function is discussed. The procedure is a modification of the Robbins-Monro procedure which is designed to approximate the zero from below. An almost sure convergence is proved and it is shown that one can guarantee that the procedure overestimate the zero only finitely many times with probability one.
Let $(X_i, Y_i)$ be $n$ independent rv's having a common bivariate distribution. When the $X_i$ are arranged in nondecreasing order as the order statistics $X_{r:n} (r = 1,2,\cdots, n)$, the $Y$-variate $Y_{\lbrack r:n\rbrack}$ paired with $X_{r:n}$ is termed the concomitant of the $r$th order statistic. The small-sample theory of the distribution and expected value of the rank $R_{r:n}$ of $Y_{\lbrack r:n\rbrack}$ is studied. In the special case of bivariate normality an…
Let $R^n$ be Euclidean $n$-space and let $O(n)$ be the group of $n \times n$ orthogonal matrices. Consider $\mathscr{F}_0 = \{f\mid f$ is a density on $R^n, f(x) = f(gx), x \in R^n, g \in O(n)\}$, and let $Q = \{q\mid q: \lbrack 0, \infty) \rightarrow \lbrack 0, \infty), q$ is nonincreasing, $\int_{R^n} q(\| x \|^2) dx = 1\}$. If $\Sigma$ is an $n \times n$ positive definite matrix, set $\mathscr{F}_1(\Sigma) = \{f|f(x) = \mid \Sigma|^{-\frac{1}{2}}q(x'\Sigma^{-1}x), q \in Q\}$…
Sufficient conditions for the Cramer-Rao inequality are formulated which do not impose any requirement on the estimator and which are independent of the choice of the densities. A hereditary property is proved for these conditions and the attainability of the lower bound is studied.
Consider the problem of estimating $x$ under the inverse linear regression model $Y_i = \alpha + \beta x_i + \varepsilon_i,\quad Z_j = \alpha + \beta x + \varepsilon_j'$ for $i = 1,\cdots, n,\cdots, j = 1,\cdots, m,\cdots,$ where $\{\varepsilon_i\}, \{\varepsilon_j'\}$ are two sequences of i.i.d. rv's with 0 means and finite variances, $\{x_i\}$ is a sequence of known constants and $\alpha, \beta, x$ are unknown parameters. For fixed $T = m + n$, this paper considers a…
The problem of classifying each of $n$ observations to one of two sub-populations is considered. The classification rule examined chooses that classification with maximum posterior probability. Limiting behavior of the rule is given and several examples are presented which show that the rule can lead to classifying all observations to the same subpopulation. Three simulation studies are reported to indicate that this extreme behavior may occur in small samples.
In the empirical Bayes approach to multiple decision problems, we obtain theorems and lemmas which can be used to obtain asymptotic optimality and rate results in any multiple decision empirical Bayes problem. Applications of these results to a classification problem, a monotone multiple decision, and a selection problem are given. In addition, a special lemma unique to the monotone multiple decision problem gives improved (exact) rate results in that case.
For the problem of empirical Bayes classification into two known probability distributions on a finite outcome space, an essentially complete class of procedures is determined. This class is proven to be minimal essentially complete if there are only two possible outcomes.
It is shown by Takeuchi and Akahira, 1974, that conditional independence together with a condition of "partial sufficiency" imply "prediction sufficiency" for loss functions not depending on the unknown parameter. We shall here prove that these conditions are necessary as well and thereby obtain a complete description, in terms of conditional expectations, of "prediction sufficiency" for loss functions not depending on the unknown parameter. It turns out that these…
Let $X_{ni}, i = 1, \cdots, n$ be i.i.d. random variables on an arbitrary measurable space $(\mathscr{X}, B)$. Suppose $\mathscr{L}(X_{ni}) = Q_{n1}, i = 1, \cdots, n$ and let $P_0$ be a fixed probability measure on $(\mathscr{X}, B)$. We consider limiting distribution theory for $U$-statistics $T_n = n^{-1} \sum_{i \neq j} Q(X_{ni}, X_{nj})$ (1) under conditions which imply the product measures $Q_n = Q_{n1} \times \cdots \times Q_{n1}, n$ times, are contiguous to the…
Measures of location differentiable at every density in the Hellinger metric are constructed in this paper. Differentiability entitles these location functionals to the label "robust," even though their influence curves need not be bounded and continuous. The latter properties are, in fact, associated with functionals differentiable in the Prokhorov metric. A Hellinger metric concept of minimax robustness of a location measure at a density shape f is developed.…
It is shown that the limiting distribution of any regular estimator of a continuous cdf on [0, 1] can be represented as a convolution of the Brownian bridge process with another distribution on C[0,1]. The result is related to Hajek's representation for limiting distributions of regular parametric estimators.
Based on a random sample from a univariate distribution with density f, this note exhibits a class of kernel estimators of the pth order derivative f(p) of f,p≧0 fixed. These estimators improve some known estimators of f(p) by weakening the conditions, sharpening the rates of convergence, or both for the properties of strong consistency, asymptotic unbiasedness and mean square consistency, each uniform on the real line.
The empirical distribution function Pn converges with probability 1 to a true distribution P in Rk, uniformly over measurable convex sets, if and only if P is a countable mixture of distributions, each of which is carried by a flat and gives zero probability to the relative boundaries of convex sets included in the flat.
The sequential estimation of p, the probability of success in a sequence of Bernoulli trials, is considered for the case where loss is taken to be symmetrized relative squared error of estimation, plus a fixed cost c per observation. Using sn/n as a terminal estimator of p, where sn is the number of successes in n trials, a heuristic rule is derived and shown to perform well for any fixed 0<p<1 as c→0. However for any fixed c>0, this…
The stopping time N of a sequential test based on an i.i.d. sequence X,X1,X2,⋯ with common distribution P is defined as the first integer n≧1 such that l1<Ln<l2 is violated, where Ln is a statistic depending only on X1,⋯,Xn. If for all l1,l2 there exist constants c>0,ρ<1 such that P(N>n)<cρn,n=1,2,⋯, then N is called exponentially bounded under P. In the contrary case P is termed…
Let a and b be r-dimensional real vectors. It is shown that if a majorizes b, then E(Πj=1rXjaj)≧E(Πj=1rXjbj) holds for nonnegative random variables X1,⋯,Xr whose joint pdf is permutation symmetric. If in addition the components of a,b are nonnegative integers, then for every Borel-measurable set A, Πj=1rP[∩i=1aj{Zi∈A}]≧Πj=1rP[∩i=1bj{Zi∈A}]…
Sequences of tests with error exp(−nA) of the first type are investigated. It is shown that the error of the second type of such a sequence of tests is bounded by exp(−nB) where B is determined by the Kullback-Leibler information distance of the hypotheses tested. The information distance between the empirical measure and the null-hypothesis on a finite partition of the sample space is proposed to use as a test statistic. A sufficient condition is given which…
Let (Xi,Yi) be n independent rv's having a common bivariate distribution. When the Xi are arranged in nondecreasing order as the order statistics Xr:n(r=1,2,⋯,n), the Y-variate Y[r:n] paired with Xr:n is termed the concomitant of the rth order statistic. The small-sample theory of the distribution and expected value of the rank Rr:n of Y[r:n] is studied. In the special case of bivariate normality an…
Let Rn be Euclidean n-space and let O(n) be the group of n×n orthogonal matrices. Consider F0={f∣f is a density on Rn,f(x)=f(gx),x∈Rn,g∈O(n)}, and let Q={q∣q:[0,∞)→[0,∞),q is nonincreasing, ∫Rnq(∥x∥2)dx=1}. If Σ is an n×n positive definite matrix, set F1(Σ)={f∣f(x)=∣Σ∣−21q(x′Σ−1x),q∈Q}…
Consider the problem of estimating x under the inverse linear regression model Yi=α+βxi+εi,Zj=α+βx+εj′ for i=1,⋯,n,⋯,j=1,⋯,m,⋯, where {εi},{εj′} are two sequences of i.i.d. rv's with 0 means and finite variances, {xi} is a sequence of known constants and α,β,x are unknown parameters. For fixed T=m+n, this paper considers a…
The problem of classifying each of n observations to one of two sub-populations is considered. The classification rule examined chooses that classification with maximum posterior probability. Limiting behavior of the rule is given and several examples are presented which show that the rule can lead to classifying all observations to the same subpopulation. Three simulation studies are reported to indicate that this extreme behavior may occur in small samples.
Large Sample Theory for U-Statistics and Tests of Fit
Let Xni,i=1,⋯,n be i.i.d. random variables on an arbitrary measurable space (X,B). Suppose L(Xni)=Qn1,i=1,⋯,n and let P0 be a fixed probability measure on (X,B). We consider limiting distribution theory for U-statistics Tn=n−1∑i=jQ(Xni,Xnj) (1) under conditions which imply the product measures Qn=Qn1×⋯×Qn1,n times, are contiguous to the…