Three stopping rules are proposed to detect a change in a normal mean, when the initial value of the mean is unknown but an estimate obtained from a training sample is available. Asymptotic approximations are given for the average run length when there is no change. Under certain hypotheses about the length of time before the change occurs and the magnitude of the change, we obtain asymptotic approximations for the expected delay in detection in terms of the corresponding…
The problem of finding admissible, asymptotically optimal compound rules is pursued in the infinite state case. The components involve the estimation of an arbitrary continuous transform of the natural parameter of a real exponential family with compact parameter space. We show that all Bayes estimators are admissible. Our main result is that any Bayes compound estimator versus a mixture of i.i.d. priors on the compound parameter is asymptotically optimal if the mixing…
Consider i.i.d. pairs $(\theta_i, X_i), i \geq 1$, where $\theta_1$ has an unknown prior distribution $\omega$ and given $\theta_1, X_1$ has distribution $P_{\theta_1}$. This setup arises naturally in the empirical Bayes problems. We put a probability (a hyperprior) on the space of all possible $\omega$ and consider the posterior mean $\hat{\omega}$ of $\omega$. We show that, under reasonable conditions, $P_{\hat{\omega}} = \int P_\theta d\hat{\omega}$ is consistent in $L_1$.…
This paper compares three methods for producing lower bounds on the minimax risk under quadratic loss. The first uses the bounds from Brown and Gajek. The second method also uses the information inequality and results in bounds which are always at least as good as those from the first method. The third method is the hardest-linear-family method described by Donoho and Liu. These methods are applied in four examples, the last of which relates to a frequently considered…
Let $X$ have a discrete density of the form $f(x) = t(x)\xi(\theta)\theta^{x_1}_1 \cdots \theta^{x_p}_p$, where $t(x)$ is nonzero on some infinite subset of $Z^p$. Consider simultaneous estimation of the $\theta_i$ under the loss $\mathscr{L}_m(\theta, \delta) = \sum^p_{i = 1} \theta^{-m}_i(\theta_i - \delta_i)^2, m \geq 0$. For integers $m \geq 1$, estimators are found which improve on the maximum likelihood estimator or uniformly minimum variance unbiased estimator. The…
Let $X_1, X_2$ be two $p \times 1$ multivariate normal random vectors and $S_1, S_2$ be two $p \times p$ Wishart matrices, where $X_1 \sim N_p(\xi, \sum_1), X_2 \sim N_p(\xi, \sum_2), S_1 \sim W_p(\sum_1, n)$ and $S_2 \sim W_p(\sum_2, n)$. We further assume that $X_1, X_2, S_1, S_2$ are stochastically independent. We wish to estimate the common mean $\xi$ with respect to the loss function $L = (\hat{\xi} - \xi)'(\sum^{-1}_1 + \sum^{-1}_2)(\hat{\xi} - \xi)$. By extending the…
Let $S_1$ and $S_2$ be two independent $p \times p$ Wishart matrices with $S_1 \sim W_p(\sum_1, n_1)$ and $S_2 \sim W_p(\sum_2, n_2)$. We wish to estimate $(\sum_1, \sum_2)$ under the loss function $L(\hat{\sum}_1, \hat{\sum}_2; \sum_1, \sum_2) = \sum_i\{\operatorname{tr}(\sum^{-1}_i \hat{\sum}_i) - \log|\sum^{-1}_i\hat{\sum}_i| - p\}$. Our approach is to first utilize the principle of invariance to narrow the class of estimators under consideration to the equivariant ones.…
This paper presents the asymptotic distribution of Ihara and Kano's noniterative estimator of the uniqueness in exploratory factor analysis. When the number of factors is overestimated, the estimator is not a continuous function of the sample covariance matrix and its asymptotic distribution is not normal, but the consistency holds. It is also shown that the first-order moment of the asymptotic distribution does not exist.
A relatively obscure eigenvalue due to Wielandt is used to give a simple derivation of the asymptotic distribution of the eigenvalues of a random symmetric matrix. The asymptotic distributions are obtained under a fairly general setting. An application of the general theory to the bootstrap distribution of the eigenvalues of the sample covariance matrix is given.
Consider a generalized linear model with response $Y$ and scalar predictor $X$. Instead of observing $X$, a surrogate $W = X + Z$ is observed, where $Z$ represents measurement error and is independent of $X$ and $Y$. The efficient score test for the absence of association depends on $m(w) = E(X\mid W = w)$ which is generally unknown. Assuming that the distribution of $Z$ is known, asymptotically efficient tests are constructed using nonparametric estimators of $m(w)$. Rates…
In the additive effects outliers (A.O.) model considered here one observes $Y_{j,n} = X_j + \upsilon_{j,n}, 0 \leq j \leq n$, where $\{X_j\}$ is the first order autoregressive [AR(1)] process with the autoregressive parameter $|\rho| < 1$. The A.O.'s $\{\upsilon_{j,n}, 0 \leq j \leq n\}$ are i.i.d. with distribution function (d.f.) $(1 - \gamma_n)I\lbrack x \geq 0\rbrack + \gamma_n L_n(x), x \in \mathbb{R}, 0 \leq \gamma_n \leq 1$, where the d.f.'s $\{L_n, n \geq 0\}$ are…
We consider asymptotic properties of the least squares estimator (LSE) in a regression model with long-memory stationary errors. First we derive a necessary and sufficient condition that the LSE be asymptotically efficient relative to the best linear unbiased estimator (BLUE). Then we derive the asymptotic distribution of the LSE under a condition on the higher-order cumulants of the white-noise process of the errors.
Since the pioneering work of Friedman and Stuetzle in 1981, projection-pursuit algorithms have attracted increasing attention. This is mainly due to their potential for overcoming or reducing difficulties arising in nonparametric regression models associated with the so-called curse of dimensionality, that is, the amount of data required to avoid an unacceptably large variance increasing rapidly with dimensionality. Subsequent work has, however, uncovered a dependence on…
A new method is presented for flexible regression modeling of high dimensional data. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data. This procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties. Unlike recursive partitioning,…
Patrizia Berti · Eugenio Regazzini · Pietro Rigo·刊期 1991-03-01 · 卷 19（1）·The Annals of Statistics已编入正刊
相容统计推断与贝叶斯定理AI 解读
Conditions are given which suffice for the assessment of a coherent inference by means of a Bayesian algorithm, i.e., a suitable extension of the classical Bayes theorem relative to a finite number of alternatives. Under some further hypotheses such inference is shown to be, in addition, coherent in the sense of Heath, Lane and Sudderth. Moreover, a characterization of coherent posteriors is provided, together with some remarks concerning finitely additive conditional…
Finite-sample replacement breakdown points are derived for different types of estimators of multivariate location and covariance matrices. The role of various equivariance properties is illustrated. The breakdown point is related to a measure of performance based on large deviations probabilities. Finally, we show that one-step reweighting preserves the breakdown point.
We consider the problem of determining whether a threshold autoregressive model fits a stationary time series significantly better than an autoregressive model does. A test statistic $\lambda$ which is equivalent to the (conditional) likelihood ratio test statistic when the noise is normally distributed is proposed. Essentially, $\lambda$ is the normalized reduction in sum of squares due to the piecewise linearity of the autoregressive function. It is shown that, under…
Let $(X, \mathscr{A})$ be a measurable space and $\{P_{\vartheta,\tau}\mid\mathscr{A}: \vartheta \in \Theta, \tau \in T\}$ a family of probability measures. Given an appropriate estimator sequence for $\vartheta$, we define a sequence of asymptotic maximum likelihood estimators for $\tau$ and give bounds for its large deviation probabilities under conditions which are natural for the application to the estimation of mixing distributions. This paper generalizes earlier…
An importance sampling method studied by Siegmund and Asmussen in the case of waiting time probabilities is extended to the mean and other functionals. The ideas involve exponential families of queues and control variates (regression), and it is found both from theory and practice that the method is dramatically better than standard tools like regenerative simulation or sample-mean estimation, not least under heavy traffic conditions.
H. W. Block · D. Chhetry · Z. Fang · A. R. Sampson·刊期 1990-12-01 · 卷 18（4）·The Annals of Statistics已编入正刊
排列上的偏序与二元经验分布的依赖序AI 解读
Three well-known partial orderings and one new one, designated by $\geq_{b_t}, t = 1, 2, 3, 4$, are defined on permutations of $\{1, 2, \cdots, n\}$ through a unified approach. Various formulations of these partial orderings are also considered. With the aid of these formulations we show that the four orderings on permutations are equivalent to positive dependence orderings defined over empirical distributions based on rank data. In particular, we show that the orderings $b_1, b_2, b_3$…
Suppose $X_1,\cdots, X_n$ are distributed according to a probability measure under which $X_1,\cdots, X_n$ are independent, $X_1 \sim F_0$, for $i = 1,\cdots, \lbrack\theta_n n\rbrack$ and $X_i \sim F^{(n)}$ for $i = \lbrack\theta_nn\rbrack + 1, \cdots, n$ where $\lbrack x\rbrack$ denotes the integer part of $x$. In this paper we consider the asymptotic efficient estimation of $\theta_n$ when the distributions are not known. Our estimator is efficient in the sense that if $F^{(n)} = F_{\eta_n}, \eta_n \rightarrow 0$…
The classical stochastic approximation problem can be regarded as choosing design points so that the responses are close to some target level in the expected squared distance. Motivated by different loss criteria, a family of stochastic approximation algorithms is proposed. This family has the same simplicity as the classical Robbins-Monro procedure does and contains the latter as a special case. Using appropriate representations and martingale limit theorems, we establish…
In the class of polynomial regression models up to degree $n$ we determine the design on $\lbrack -1, 1\rbrack$ that maximizes a product of $n + 1$ determinants of information matrices weighted with a prior $\beta$, where the $l$-th information matrix corresponds to a polynomial regression model of degree $l$, for $l = 0, 1, \cdots, n$. The designs are calculated using canonical moments. We identify a special class of priors $\beta(z)$ depending on one real parameter $z$ so…
A linear process is represented as a driving white noise convolved with a system response sequence. The concept of natural peakedness of a system response sequence is defined and its properties are investigated. Utilizing natural peakedness, the convergence theory of maximum standardized cumulant deconvolution is established and the uniqueness theorem of non-Gaussian linear process representations is proved. In addition, autoregressive models on a countable abelian group are…
Consider i.i.d. pairs (θi,Xi),i≥1, where θ1 has an unknown prior distribution ω and given θ1,X1 has distribution Pθ1. This setup arises naturally in the empirical Bayes problems. We put a probability (a hyperprior) on the space of all possible ω and consider the posterior mean ω^ of ω. We show that, under reasonable conditions, Pω^=∫Pθdω^ is consistent in L1.…
Let X have a discrete density of the form f(x)=t(x)ξ(θ)θ1x1⋯θpxp, where t(x) is nonzero on some infinite subset of Zp. Consider simultaneous estimation of the θi under the loss Lm(θ,δ)=∑i=1pθi−m(θi−δi)2,m≥0. For integers m≥1, estimators are found which improve on the maximum likelihood estimator or uniformly minimum variance unbiased estimator. The…
Let X1,X2 be two p×1 multivariate normal random vectors and S1,S2 be two p×p Wishart matrices, where X1∼Np(ξ,∑1),X2∼Np(ξ,∑2),S1∼Wp(∑1,n) and S2∼Wp(∑2,n). We further assume that X1,X2,S1,S2 are stochastically independent. We wish to estimate the common mean ξ with respect to the loss function L=(ξ^−ξ)′(∑1−1+∑2−1)(ξ^−ξ). By extending the…
Let S1 and S2 be two independent p×p Wishart matrices with S1∼Wp(∑1,n1) and S2∼Wp(∑2,n2). We wish to estimate (∑1,∑2) under the loss function L(∑^1,∑^2;∑1,∑2)=∑i{tr(∑i−1∑^i)−log∣∑i−1∑^i∣−p}. Our approach is to first utilize the principle of invariance to narrow the class of estimators under consideration to the equivariant ones.…
Consider a generalized linear model with response Y and scalar predictor X. Instead of observing X, a surrogate W=X+Z is observed, where Z represents measurement error and is independent of X and Y. The efficient score test for the absence of association depends on m(w)=E(X∣W=w) which is generally unknown. Assuming that the distribution of Z is known, asymptotically efficient tests are constructed using nonparametric estimators of m(w). Rates…
In the additive effects outliers (A.O.) model considered here one observes Yj,n=Xj+υj,n,0≤j≤n, where {Xj} is the first order autoregressive [AR(1)] process with the autoregressive parameter ∣ρ∣<1. The A.O.'s {υj,n,0≤j≤n} are i.i.d. with distribution function (d.f.) (1−γn)I[x≥0]+γnLn(x),x∈R,0≤γn≤1, where the d.f.'s {Ln,n≥0} are…
We consider the problem of determining whether a threshold autoregressive model fits a stationary time series significantly better than an autoregressive model does. A test statistic λ which is equivalent to the (conditional) likelihood ratio test statistic when the noise is normally distributed is proposed. Essentially, λ is the normalized reduction in sum of squares due to the piecewise linearity of the autoregressive function. It is shown that, under…
Let (X,A) be a measurable space and {Pϑ,τ∣A:ϑ∈Θ,τ∈T} a family of probability measures. Given an appropriate estimator sequence for ϑ, we define a sequence of asymptotic maximum likelihood estimators for τ and give bounds for its large deviation probabilities under conditions which are natural for the application to the estimation of mixing distributions. This paper generalizes earlier…
Three well-known partial orderings and one new one, designated by ≥bt,t=1,2,3,4, are defined on permutations of {1,2,⋯,n} through a unified approach. Various formulations of these partial orderings are also considered. With the aid of these formulations we show that the four orderings on permutations are equivalent to positive dependence orderings defined over empirical distributions based on rank data. In particular, we show that the orderings b1,b2,b3…
Suppose X1,⋯,Xn are distributed according to a probability measure under which X1,⋯,Xn are independent, X1∼F0, for i=1,⋯,[θnn] and Xi∼F(n) for i=[θnn]+1,⋯,n where [x] denotes the integer part of x. In this paper we consider the asymptotic efficient estimation of θn when the distributions are not known. Our estimator is efficient in the sense that if F(n)=Fηn,ηn→0…
Nonlinear Stochastic Approximation Procedures for Lp Loss Functions
A Generalization of D- and D1-Optimal Designs in Polynomial Regression
In the class of polynomial regression models up to degree n we determine the design on [−1,1] that maximizes a product of n+1 determinants of information matrices weighted with a prior β, where the l-th information matrix corresponds to a polynomial regression model of degree l, for l=0,1,⋯,n. The designs are calculated using canonical moments. We identify a special class of priors β(z) depending on one real parameter z so…