We investigate the problem of estimating the mean vector $\mathbf{\theta}$ of a multivariate normal distribution with covariance matrix equal to $\sigma^2\mathbf{I}_p, \sigma^2$ unknown, and loss $\|\delta - \mathbf{\theta}\|^2/\sigma^2$. We first find a class of minimax estimators for this problem which enlarges a class given by Baranchik. This result is then used to show that for sufficiently large sample sizes (which never need exceed 4) proper Bayes minimax estimators…
The complete asymptotic expansion in powers of $N^{-\frac{1}{2}}$ for the distribution of the two-sample statistic $V_{NN}$ is derived under the null hypothesis. The proof is based on methods developed by Kemperman (1959).
The problem of estimating the zero of a regression function by means of Robbins Monro type of stochastic approximation procedures is discussed. Optimality of the procedures is defined in terms of asymptotic variance. The discussion is restricted to the case of identically distributed errors. In that case we suggest transforming the observed random variables in order to minimize the asymptotic variance of the estimators. The optimal transformation turns out to depend on the…
The general stochastic approximation procedure: $X_{n+1} = X_n - a_n c_n^{-1}h(Y_n),\quad n = 1,2, \cdots$ is considered, where $h$ is a Borel measurable transformation on the random observations $Y_n$. Under some mild requirements on $h$ and on the error random variables, the asymptotic properties, the a.s. convergence and the asymptotic normality are studied. The analysis is confined to the case where the error random variables are (conditionally) distributed according to…
In this paper the minimax estimators of a cumulative distribution function $F$ is obtained for four types of loss functions. The result is quite general in that no restrictions are imposed on the unknown $F$. Moreover, the estimates do not depend upon the weight function used in the definition of the loss functions. It is also shown that the sample distribution function is minimax under one of these types of loss functions.
Let $p$ be the probability of any event in repeated independent trials. Sequential tests of the composite hypothesis $p \leqq p_0$ against the composite hypothesis $p > p_0$ are proposed, which asymptotically minimize the maximum risk when the cost of experimentation tends to zero, if the loss depends only on $p$ and satisfies some natural regularity conditions. Asymptotic power, expected sample size and risk of the tests are also given.
The asymptotic behavior is given for the error rates and ASN of the Wald SPRT and of invariant sequential tests. An asymptotic justification of Bhate's conjecture is also provided for invariant sequential tests. Expressions are obtained for the asymptotic relative efficiency of the Wald SPRT as compared with the corresponding best non-sequential test.
Let $F_n(x)$ be the empirical distribution function based on a random sample of size $n$ from a continuous symmetric distribution with center $\theta$. As a nonparametric estimator of $\theta$, we propose $a^\ast$ where $a^\ast$ is chosen so as to minimize the function $h$ where $h(a) = \max_x |F_n(x) + F_n((2a - x)^-) - 1|$. In this paper we present an algorithm for constructing the interval of all $a$ which minimize $h$. We show that if $a^\ast$ is chosen as the center of…
In this paper, the $c$-sample location problem with ordered or restricted alternatives is considered. Linear combinations of Chernoff-Savage type two-sample statistics computed among the $c(c - 1)/2$ pairs of samples are proposed as test statistics. It is shown that for each linear combination of two-sample statistics there is another linear combination, using only the $c - 1$ two-sample statistics based on adjacent samples as determined by the alternative, which has the…
The rank statistics $S_{\Delta N} = N^{-1} \sum^N_{i=1} c_{Ni} R^\Delta_{Ni}$, with $R^Delta_{Ni}$ being the rank of $X_{Ni} + \Delta d_{Ni}, i = 1, 2, \cdots, N$ and $X_{N1}, \cdots, X_{NN}$ being the random sample from the basic distribution with density function $f$, are considered as a random process with $\Delta$ in the role of parameter. Under some assumptions on $C_{Ni}$'s, $d_{Ni}$'s and on the underlying distribution, it is proved that the process $\{S_{\Delta N} - S_{0N} - ES_{\Delta N}; 0 \leqq \Delta \leqq 1\}$…
This paper contains an Edgeworth-type expansion for the distribution of a minimum contrast estimator, and expansions suitable for the computation of critical regions of prescribed error (type one) as well as confidence intervals of prescribed confidence coefficient. Furthermore, it is shown that, for one-sided alternatives, the test based on the maximum likelihood estimator as well as the test based on the derivative of the log-likelihood function is uniformly most powerful…
The theory of rank tests has been developed primarily for continuous random variables. Recently the asymptotic theory of linear rank tests has been extended to include purely discrete random variables under the null hypothesis of randomness (including the two-sample and $k$-sample problems) and under contiguous alternatives, for the two methods of assigning scores known as the average scores method and the randomized ranks method. In this paper the theory of rank tests is…
Let $Z_n = \max_{1\leqq j\leqq n} X_j$ where $\{X_n: 1 \leqq n < \infty\}$ is a Gaussian process. Some known limit theorems for $Z_n$ when $\{X_n\}$ is stationary are extended to the nonstationary case.
It is shown that under certain conditions, the maximum likelihood and the minimum variance unbiased estimators of a positive integral power of the natural parameter in an exponential family have the same asymptotic distribution.
The classification statistic $W$ is used to classify an observasion as coming from one of two multivariate normal populations with common covariance matrix and different means when these parameters are estimated from two samples, one from each population. The distribution of $W$ depends on the Mahalanobis distance between the populations, $\alpha$. When the sample sizes approach infinity, the limiting distribution of $(W - \frac{1}{2}\alpha)/\alpha^{\frac{1}{2}}$ is the…
A class of statistics is considered, which are based on order-statistics and have properties analogous to Student's $t$. They can be used to estimate a required quantile of a random variable or to test hypotheses about a quantile; they are simple to compute, and can be calculated when not all sample values are available, e.g. for censored samples. The limiting distributions of these statistics are derived and shown to be independent of the distribution of the underlying…
The large-sample distributions of the maximum-likelihood estimates for the index, skewness, scale, and location parameters (respectively $\alpha, \beta, c$, and $\delta$) of a stable distribution are studied. It is shown that if both $\alpha$ and $\delta$ are unknown, then the likelihood function $L$ will have no maximum within $0 < \alpha \leqq 2, -\infty < \delta < \infty$, but that $L(\alpha, \delta) \rightarrow \infty$ as $(\alpha, \delta) \rightarrow (0, x_k)$ where $x_k$…
$f(x)$ is a uniformly continuous density which equals zero for negative values of $x$, has a right-hand derivative equal to $\alpha$ at $x = 0$, where $0 < \alpha < \infty$, and satisfies certain regularity conditions. $X_1,\cdots, X_n$ are independent random variables with the common density $f(x - \theta), \theta$ an unknown parameter. Let $\hat{\theta}_n$ denote the maximum likelihood estimator of $\theta$, and define $\alpha_n$ by the equation $2\alpha_n^2 = \alpha n \log n$…
The confidence sets for linear functions $\mu + \lambda\sigma^2$ of the mean $\mu$ and variance $\sigma^2$ of a normal distribution, defined in terms of the uniformly most powerful unbiased level $\alpha$ tests of hypotheses of form $H_0(\lambda, m): \mu + \lambda\sigma^2 = m$ against the two-sided alternative $H_1(\lambda, m): \mu + \lambda\sigma^2 \neq m$ for $-\infty < m < \infty$, for fixed $\alpha$ and $\lambda$, are shown to be intervals if the number of degrees of…
Gontcharov obtained the joint probability generating function for the numbers of runs of all lengths, both of successes and failures, in a Bernoulli sequence. This is here generalized to a class of regenerative binary Markov processes. For an allied class of Markov processes, the probability generating function is obtained for a "total score" defined in terms of runs of successes only, and asymptotic formulas are derived for the expectation and variance of the score.
This paper is concerned with maximum likelihood estimates for a large class of families of unimodal densities. The existence of measurable maximum likelihood estimates and the consistency of asymptotic maximum likelihood estimates are proved. By counterexamples it is shown that the conditions which are sufficient for consistency cannot be removed without compensation.
A complexity approach is used to decide whether or not the mean of a sequence of independent identically distributed random variables lies in an arbitrary specified countable subset of the real line. A procedure is described that makes only a finite number of mistakes with probability one. This leads to some speculations on inference of the laws of physics and the computability of the physical constants.
The purpose of this paper is to find asymptotically optimal Bayes sequential procedures for estimating a function $g(\theta_1, \theta_2,\cdots, \theta_k)$ when there are $k$ experiments $E_1, E_2,\cdots, E_k$ and the performance of the experiment $E_i$ conducts to the observation of a random variable whose distribution depends on the vector parameter $\theta_i$. The term asymptotical refers here to the cost of experimentation tending to zero. The methods used are a…
In this paper Laws of the Iterated Logarithm for maximums of absolute values of partial sums of permuted random variables are derived under conditions that are the same as or similar to conditions used by Kolmogorov, Hartman and Wintner, Petrov and Csaki in deriving Laws of the Iterated Logarithm for sums of random variables or semimartingales. These results are then applied to obtain logarithmic convergence rates for estimators of non-decreasing regression functions and…
We investigate the problem of estimating the mean vector θ of a multivariate normal distribution with covariance matrix equal to σ2Ip,σ2 unknown, and loss ∥δ−θ∥2/σ2. We first find a class of minimax estimators for this problem which enlarges a class given by Baranchik. This result is then used to show that for sufficiently large sample sizes (which never need exceed 4) proper Bayes minimax estimators…
The Asymptotic Expansion of the Distribution of the Goodness of Fit Statistic VNN
The complete asymptotic expansion in powers of N−21 for the distribution of the two-sample statistic VNN is derived under the null hypothesis. The proof is based on methods developed by Kemperman (1959).
The general stochastic approximation procedure: Xn+1=Xn−ancn−1h(Yn),n=1,2,⋯ is considered, where h is a Borel measurable transformation on the random observations Yn. Under some mild requirements on h and on the error random variables, the asymptotic properties, the a.s. convergence and the asymptotic normality are studied. The analysis is confined to the case where the error random variables are (conditionally) distributed according to…
In this paper the minimax estimators of a cumulative distribution function F is obtained for four types of loss functions. The result is quite general in that no restrictions are imposed on the unknown F. Moreover, the estimates do not depend upon the weight function used in the definition of the loss functions. It is also shown that the sample distribution function is minimax under one of these types of loss functions.
Let p be the probability of any event in repeated independent trials. Sequential tests of the composite hypothesis p≦p0 against the composite hypothesis p>p0 are proposed, which asymptotically minimize the maximum risk when the cost of experimentation tends to zero, if the loss depends only on p and satisfies some natural regularity conditions. Asymptotic power, expected sample size and risk of the tests are also given.
Let Fn(x) be the empirical distribution function based on a random sample of size n from a continuous symmetric distribution with center θ. As a nonparametric estimator of θ, we propose a∗ where a∗ is chosen so as to minimize the function h where h(a)=maxx∣Fn(x)+Fn((2a−x)−)−1∣. In this paper we present an algorithm for constructing the interval of all a which minimize h. We show that if a∗ is chosen as the center of…
In this paper, the c-sample location problem with ordered or restricted alternatives is considered. Linear combinations of Chernoff-Savage type two-sample statistics computed among the c(c−1)/2 pairs of samples are proposed as test statistics. It is shown that for each linear combination of two-sample statistics there is another linear combination, using only the c−1 two-sample statistics based on adjacent samples as determined by the alternative, which has the…
The rank statistics SΔN=N−1∑i=1NcNiRNiΔ, with RDeltaNi being the rank of XNi+ΔdNi,i=1,2,⋯,N and XN1,⋯,XNN being the random sample from the basic distribution with density function f, are considered as a random process with Δ in the role of parameter. Under some assumptions on CNi's, dNi's and on the underlying distribution, it is proved that the process {SΔN−S0N−ESΔN;0≦Δ≦1}…
Rank Tests for One Sample, Two Samples, and k samples Without the Assumption of a Continuous Distribution Function
The theory of rank tests has been developed primarily for continuous random variables. Recently the asymptotic theory of linear rank tests has been extended to include purely discrete random variables under the null hypothesis of randomness (including the two-sample and k-sample problems) and under contiguous alternatives, for the two methods of assigning scores known as the average scores method and the randomized ranks method. In this paper the theory of rank tests is…
Let Zn=max1≦j≦nXj where {Xn:1≦n<∞} is a Gaussian process. Some known limit theorems for Zn when {Xn} is stationary are extended to the nonstationary case.
An Asymptotic Expansion of the Distribution of the Studentized Classification Statistic W1
The classification statistic W is used to classify an observasion as coming from one of two multivariate normal populations with common covariance matrix and different means when these parameters are estimated from two samples, one from each population. The distribution of W depends on the Mahalanobis distance between the populations, α. When the sample sizes approach infinity, the limiting distribution of (W−21α)/α21 is the…
Limiting Distributions of Statistics Similar to Student's t
A class of statistics is considered, which are based on order-statistics and have properties analogous to Student's t. They can be used to estimate a required quantile of a random variable or to test hypotheses about a quantile; they are simple to compute, and can be calculated when not all sample values are available, e.g. for censored samples. The limiting distributions of these statistics are derived and shown to be independent of the distribution of the underlying…
The large-sample distributions of the maximum-likelihood estimates for the index, skewness, scale, and location parameters (respectively α,β,c, and δ) of a stable distribution are studied. It is shown that if both α and δ are unknown, then the likelihood function L will have no maximum within 0<α≦2,−∞<δ<∞, but that L(α,δ)→∞ as (α,δ)→(0,xk) where xk…
f(x) is a uniformly continuous density which equals zero for negative values of x, has a right-hand derivative equal to α at x=0, where 0<α<∞, and satisfies certain regularity conditions. X1,⋯,Xn are independent random variables with the common density f(x−θ),θ an unknown parameter. Let θ^n denote the maximum likelihood estimator of θ, and define αn by the equation 2αn2=αnlogn…
The confidence sets for linear functions μ+λσ2 of the mean μ and variance σ2 of a normal distribution, defined in terms of the uniformly most powerful unbiased level α tests of hypotheses of form H0(λ,m):μ+λσ2=m against the two-sided alternative H1(λ,m):μ+λσ2=m for −∞<m<∞, for fixed α and λ, are shown to be intervals if the number of degrees of…
The purpose of this paper is to find asymptotically optimal Bayes sequential procedures for estimating a function g(θ1,θ2,⋯,θk) when there are k experiments E1,E2,⋯,Ek and the performance of the experiment Ei conducts to the observation of a random variable whose distribution depends on the vector parameter θi. The term asymptotical refers here to the cost of experimentation tending to zero. The methods used are a…