We study sequential detection of a change-point using the generalized likelihood ratio statistic. For the special case of detecting a change in a normal mean with known variance, we give approximations to the average run lengths and compare our procedure to standard CUSUM tests and combined CUSUM-Shewhart tests. Several examples indicating extensions to problems involving multiple parameters are discussed.
Asymptotic normality of the difference between the number of subjects assigned to a treatment and the desired number to be assigned is established for allocation rules which use Eisele's biased coin design. Subject responses are assumed to be independent random variables from standard exponential families. In the proof, it is shown that the difference may be magnified by appropriate constants so that the magnified difference is nearly a martingale. An application to the…
We consider the problem of estimating a pdf $f$ from samples $X_1, X_2, \ldots, X_n$ of a random variable with pdf $\mathscr{K}f$, where $\mathscr{K}$ is a compact integral operator. We employ a maximum smoothed likelihood formalism inspired by a nonlinearly smoothed version of the EMS algorithm of Silverman, Jones, Wilson and Nychka. We show that this nonlinearly smoothed algorithm is itself an EM algorithm, which helps explain the strong convergence properties of the…
The frailty model is a generalization of Cox's proportional hazards model which includes a random effect. Nielsen, Gill, Andersen and Sorensen (1992) proposed an EM algorithm to estimate the cumulative baseline hazard and the variance of the random effect. Here the asymptotic distribution of the estimators is given along with a consistent estimator of the asymptotic variance.
R. J. Carroll · R. K. Knickerbocker · C. Y. Wang·刊期 1995-02-01 · 卷 23（1）·The Annals of Statistics已编入正刊
协变量含测量误差的半参数回归模型中的降维AI 解读
We consider a semiparametric estimation method for general regression models when some of the predictors are measured with error. The technique relies on a kernel regression of the "true" covariate on all the observed covariates and surrogates. This requires a nonparametric regression in as many dimensions as there are covariates and surrogates. The usual theory copes with such higher-dimensional problems by using higher-order kernels, but this is unrealistic for most…
We consider Bahadur-Kiefer representations for smoothed quantile processes. We prove that the asymptotics of the distance between smoothed empirical and quantile processes can be completely different from that of the unsmoothed ones. We obtain a complete characterization of the possible limits.
Let $\mathbf{X}$ have a multivariate, $p$-dimensional normal distribution $(p \geq 2)$ with unknown mean $\mathbf{\mu}$ and known, nonsingular covariance $\mathbf{\Sigma}$. Consider testing $H_0: \mathbf{b}'_i\mathbf{\mu} \leq 0$, for some $i = 1, \ldots, k$, versus $H_1: \mathbf{b}'_i\mathbf{\mu} > 0$, for all $i = 1, \ldots, k$, where $\mathbf{b}_1, \ldots, \mathbf{b}_k, k \geq 2$, are known vectors that define the hypotheses. For any $0 < \alpha < 1/2$, we construct a…
In 1987 Cheng determined $\phi_p$-optimal designs for linear regression (without intercept) over the $n$-dimensional unit cube $\lbrack 0, 1\rbrack^n$ for $-\infty \leq p \leq 1$. These are uniform distributions on the vertices with a fixed number of entries equal to unity, and mixtures of neighboring such designs. In 1989 Pukelsheim showed that this class of designs is essentially complete and that the corresponding class of moment matrices is minimally complete, with…
In the usual linear regression model we investigate the geometric structure of a class of minimax optimality criteria containing Elfving's minimax and Kiefer's $\phi_p$-criteria as special cases. It is shown that the optimal designs with respect to these criteria are also optimal for $A'\theta$, where $A$ is any inball vector (in an appropriate norm) of a generalized Elfving set. The results explain the particular role of the $A$- and $E$-optimality criterion and are applied…
Let $\varphi$ be a smooth function of $k + 2$ variables. We shall investigate in this paper the rates of convergence of estimators of $T(f) = \int\varphi(f(x), f'(x), \ldots, f^{(k)}(x), x) dx$ when $f$ belongs to some class of densities of smoothness $s$. We prove that, when $s \geq 2k + \frac{1}{4}$, one can define an estimator $\hat{T}_n$ of $T(f)$, based on $n$ i.i.d. observations of density $f$ on the real line, which converges at the semiparametric rate $1/ \sqrt n$.…
Variable window width kernel density estimators, with the width varying proportionally to the square root of the density, have been thought to have superior asymptotic properties. The rate of convergence has been claimed to be as good as those typical for higher-order kernels, which makes the variable width estimators more attractive because no adjustment is needed to handle the negativity usually entailed by the latter. However, in a recent paper, Terrell and Scott show…
In finite mixture models, we establish the best possible rate of convergence for estimating the mixing distribution. We find that the key for estimating the mixing distribution is the knowledge of the number of components in the mixture. While a $\sqrt n$-consistent rate is achievable when the exact number of components is known, the best possible rate is only $n^{-1/4}$ when it is unknown. Under a strong identifiability condition, it is shown that this rate is reached by…
Paul D. Valz · A. Ian McLeod · Mary E. Thompson·刊期 1995-02-01 · 卷 23（1）·The Annals of Statistics已编入正刊
含并列排名的 Kendall 得分：累积量生成函数与尾概率近似AI 解读
Robillard's approach to obtaining an expression for the cumulant generating function of the null distribution of Kendall's $S$-statistic, when one ranking is tied, is extended to the general case where both rankings are tied. An expression is obtained for the cumulant generating function and it is used to provide a direct proof of the asymptotic normality of the standardized score, $S/ \sqrt{\operatorname{Var}(S)}$, when both rankings are tied. The third cumulant of $S$ is…
Suppose we observe a uniformly ergodic Markov chain with unknown transition distribution. The empirical estimator for a linear functional of the (invariant) joint distribution of two successive observations is defined using the pairs of successive observations. Its efficiency is proved using a martingale approximation. As a corollary we show efficiency of the empirical joint distribution function in the sense of a functional convolution theorem.
In this paper we propose a Capon-type estimator for the spectrum of a stationary time series. This estimator may be viewed as an alternative to classical periodogram-based estimators. Its advantage is that it copes with the "leakage effect" by using implicitly automatic adaptive windowing. We show its asymptotic equivalence to a random variable which is a quadratic form in the observations, thus obtaining the asymptotic normality of the Capon estimator. We also study its…
Verifiable conditions are given for the validity of formal Edgeworth expansions for the distribution of sums $X_1 + \cdots + X_n$, where $X_i = F(Z_i, \ldots, Z_{i + p - 1})$ and $Z_1, Z_2, \ldots$ is a strict sense stationary sequence that can be written as $Z_j = g(\varepsilon_{j - k}: k \geq 0)$ with an $\operatorname{iid}$ sequence $(\varepsilon_i)$ of innovations. These models include nonlinear functions of ARMA processes $(Z_i)$ as well as certain nonlinear AR…
This paper studies the weak convergence of the sequential empirical process $\widehat{K}_n$ of the estimated residuals in ARMA$(p, q)$ models when the errors are independent and identically distributed. It is shown that, under some mild conditions, $\widehat{K}_n$ converges weakly to a Kiefer process. The weak convergence is discussed for both finite and infinite variance time series models. An application to a change-point problem is considered.
Recent work of several authors has focussed on first-order properties (e.g., consistency) of general bootstrap algorithms, where the numbers of times that data values are resampled form an exchangeable sequence. In the present paper we develop second-order properties of such algorithms, in a very general setting. Performance is discussed in the context of distribution estimation, and formulae for higher-order moments and cumulants are developed. Arguing thus, necessary and…
Random coefficient regression models are important in modeling heteroscedastic multivariate linear regression in econometrics. The analysis of panel data is one example. In statistics, the random and mixed effects models of ANOVA, deconvolution models and affine mixture models are all special cases of random coefficient regression. Some inferential problems, such as constructing prediction regions for the modeled response, require a good nonparametric estimator of the…
A new criterion is proposed for the evaluation of variable selection procedures in multiple regression. This criterion, which we call the risk inflation, is based on an adjustment to the risk. Essentially, the risk inflation is the maximum increase in risk due to selecting rather than knowing the "correct" predictors. A new variable selection procedure is obtained which, in the case of orthogonal predictors, substantially improves on AIC, $C_p$ and BIC and is close to…
Arthur Cohen · J. H. B. Kemperman · H. B. Sackrowitz·刊期 1994-12-01 · 卷 22（4）·The Annals of Statistics已编入正刊
指数族回归中的无偏检验AI 解读
Let $(X_{ij}, \mathbf{z}_i), i = 1, 2, \ldots, k, j = 1, 2, \ldots, n_i$, be independent observations such that $\mathbf{z}_i$ is a fixed $r \times 1$ vector $\lbrack r$ can be 0 (no $\mathbf{z}$'s observed) or $1, 2, \ldots, k - 1\rbrack$, and $X_{ij}$ is distributed according to a one-parameter exponential family which is log concave with natural parameter $\theta_i$. We test the hypothesis that $\theta = \mathbf{Z}\beta$, where $\theta = (\theta_1, \ldots, \theta_k)', \mathbf{Z}$…
Stochastic regression models of the form $y_i = f_i(\theta) + \varepsilon_i$, where the random disturbances $\varepsilon_i$ form a martingale difference sequence with respect to an increasing sequence of $\sigma$-fields $\{\mathcal{G}_i\}$ and $f_i$ is a random $\mathcal{G}_{i - 1}$-measurable function of an unknown parameter $\theta$, cover a broad range of nonlinear (and linear) time series and stochastic process models. Herein strong consistency and asymptotic normality…
Suppose one observes independent samples of size $n$ from both the mixture density $\int p(x \mid z) d\eta(z)$ and from the distribution $\eta$. The kernel $p(x \mid z)$ is known. We show asymptotic normality and efficiency of the maximum likelihood estimator for $\eta$.
Hidden Markov models are today widespread for modeling of various phenomena. It has recently been shown by Leroux that the maximum-likelihood estimate (MLE) of the parameters of a such a model is consistent, and local asymptotic normality has been proved by Bickel and Ritov. In this paper we propose a new class of estimates which are consistent, asymptotically normal and almost as good as the MLE.
We consider the problem of estimating a pdf f from samples X1,X2,…,Xn of a random variable with pdf Kf, where K is a compact integral operator. We employ a maximum smoothed likelihood formalism inspired by a nonlinearly smoothed version of the EMS algorithm of Silverman, Jones, Wilson and Nychka. We show that this nonlinearly smoothed algorithm is itself an EM algorithm, which helps explain the strong convergence properties of the…
Let X have a multivariate, p-dimensional normal distribution (p≥2) with unknown mean μ and known, nonsingular covariance Σ. Consider testing H0:bi′μ≤0, for some i=1,…,k, versus H1:bi′μ>0, for all i=1,…,k, where b1,…,bk,k≥2, are known vectors that define the hypotheses. For any 0<α<1/2, we construct a…
In 1987 Cheng determined ϕp-optimal designs for linear regression (without intercept) over the n-dimensional unit cube [0,1]n for −∞≤p≤1. These are uniform distributions on the vertices with a fixed number of entries equal to unity, and mixtures of neighboring such designs. In 1989 Pukelsheim showed that this class of designs is essentially complete and that the corresponding class of moment matrices is minimally complete, with…
In the usual linear regression model we investigate the geometric structure of a class of minimax optimality criteria containing Elfving's minimax and Kiefer's ϕp-criteria as special cases. It is shown that the optimal designs with respect to these criteria are also optimal for A′θ, where A is any inball vector (in an appropriate norm) of a generalized Elfving set. The results explain the particular role of the A- and E-optimality criterion and are applied…
Let φ be a smooth function of k+2 variables. We shall investigate in this paper the rates of convergence of estimators of T(f)=∫φ(f(x),f′(x),…,f(k)(x),x)dx when f belongs to some class of densities of smoothness s. We prove that, when s≥2k+41, one can define an estimator T^n of T(f), based on n i.i.d. observations of density f on the real line, which converges at the semiparametric rate 1/n.…
In finite mixture models, we establish the best possible rate of convergence for estimating the mixing distribution. We find that the key for estimating the mixing distribution is the knowledge of the number of components in the mixture. While a n-consistent rate is achievable when the exact number of components is known, the best possible rate is only n−1/4 when it is unknown. Under a strong identifiability condition, it is shown that this rate is reached by…
Robillard's approach to obtaining an expression for the cumulant generating function of the null distribution of Kendall's S-statistic, when one ranking is tied, is extended to the general case where both rankings are tied. An expression is obtained for the cumulant generating function and it is used to provide a direct proof of the asymptotic normality of the standardized score, S/Var(S), when both rankings are tied. The third cumulant of S is…
Verifiable conditions are given for the validity of formal Edgeworth expansions for the distribution of sums X1+⋯+Xn, where Xi=F(Zi,…,Zi+p−1) and Z1,Z2,… is a strict sense stationary sequence that can be written as Zj=g(εj−k:k≥0) with an iid sequence (εi) of innovations. These models include nonlinear functions of ARMA processes (Zi) as well as certain nonlinear AR…
This paper studies the weak convergence of the sequential empirical process Kn of the estimated residuals in ARMA(p,q) models when the errors are independent and identically distributed. It is shown that, under some mild conditions, Kn converges weakly to a Kiefer process. The weak convergence is discussed for both finite and infinite variance time series models. An application to a change-point problem is considered.
A new criterion is proposed for the evaluation of variable selection procedures in multiple regression. This criterion, which we call the risk inflation, is based on an adjustment to the risk. Essentially, the risk inflation is the maximum increase in risk due to selecting rather than knowing the "correct" predictors. A new variable selection procedure is obtained which, in the case of orthogonal predictors, substantially improves on AIC, Cp and BIC and is close to…
Let (Xij,zi),i=1,2,…,k,j=1,2,…,ni, be independent observations such that zi is a fixed r×1 vector [r can be 0 (no z's observed) or 1,2,…,k−1], and Xij is distributed according to a one-parameter exponential family which is log concave with natural parameter θi. We test the hypothesis that θ=Zβ, where θ=(θ1,…,θk)′,Z…
Stochastic regression models of the form yi=fi(θ)+εi, where the random disturbances εi form a martingale difference sequence with respect to an increasing sequence of σ-fields {Gi} and fi is a random Gi−1-measurable function of an unknown parameter θ, cover a broad range of nonlinear (and linear) time series and stochastic process models. Herein strong consistency and asymptotic normality…
Suppose one observes independent samples of size n from both the mixture density ∫p(x∣z)dη(z) and from the distribution η. The kernel p(x∣z) is known. We show asymptotic normality and efficiency of the maximum likelihood estimator for η.