We consider in this paper the estimation problems for both orders and coefficients of linear feedback control systems, described by ARMAX models. The estimation algorithms are inspired by the Hannan-Rissanen method used for the estimation of stationary ARMA models, while the convergence analyses are based on limit theorems for both double array martingales and nonnegative supermartingales, and on techniques of stochastic Lyapunov functions. Traditionally used assumptions,…
A method for robust nonparametric regression is discussed. We consider kernel $M$-estimates of the regression function using Huber's $\psi$-function and extend results of Hardle and Gasser to the case of random designs. A practical adaptive procedure is proposed consisting of simultaneously minimising a cross-validatory criterion with respect to both the smoothing parameter and a robustness parameter occurring in the $\psi$-function. This method is shown to possess a…
The problem of automatic bandwidth selection for a kernel regression estimate is studied. Since the bandwidth considerably affects the features of the estimated curve, it is important to understand the behavior of bandwidth selection procedures. The bandwidth estimate considered here is the minimizer of Mallows' criterion. Though it was established that the bandwidth estimate is asymptotically normal, it is well recognized that the rate of convergence is extremely slow. In…
A general approach to the first order asymptotic analysis of penalized likelihood and related estimators is described. The method gives expansions for the systematic and random error. Asymptotic convergence rates in a family of spectral norms are obtained. The theory applies to a broad range of function estimation problems including nonparametric density, hazard and generalized regression curve estimation. Some examples are provided.
We consider the consistency and weak convergence of $S$-estimators in the linear regression model. Sufficient conditions for consistency with varying dimension are given which are sufficiently weak to cover, for example, polynomial trends and i.i.d. carriers. A weak convergence theorem for the Hampel-Rousseeuw least median of squares estimator is obtained, and it is shown under rather general conditions that the correct norming factor is $n^{1/3}$. Finally, the asymptotic…
Biological models for the independent action of two drugs imply that if the drugs are administered in various combinations of doses, then the corresponding probabilities of response must satisfy certain inequalities if the drugs are acting independently. The hypothesis that the probabilities do satisfy these inequalities can be tested using the likelihood ratio test or using the bootstrap test proposed by Wahrendorf and Brown in 1980. In the simplest dose design, only one…
Any multiresponse estimation experiment requires a decision about the number of observations to be taken. If the covariance is unknown, no fixed-sample-size procedure can guarantee that the joint confidence region will have an assigned shape and level. Double-sampling procedures use a preliminary sample of size $m$ to determine the minimum number of additional observations needed to achieve a prescribed accuracy and coverage probability for the parameter estimates. The…
This paper presents lower bounds, derived from the information inequality, for the Bayes risk under scaled quadratic loss. Some numerical results are also presented which give some idea concerning the precision of these bounds. An appendix contains a proof of the information inequality without conditions on the estimator. This result is a direct extension of an earlier result of Fabian and Hannan.
A bivariate Weibull model is proposed for censored survival data when there may be dependence between the survival and censoring random variables. The model is based on marginal transformations of the Pickands class of bivariate exponential distributions. Assuming the model to hold, the joint distribution of survival and censoring times is shown to be identifiable, and the maximum likelihood estimator of the parameters is shown to be consistent.
The correspondence between a survival function and its hazard or failure-rate is a central idea in survival analysis and in the theory of counting processes. This correspondence is shown to be a special case of a more general correspondence between multiplicative and additive matrix-valued measures on the real line. Additive integration of the survival function produces the hazard, while the multiplicative integral, or so-called product-integral, of the hazard yields the…
Let $\{X_n\}$ be the branching process with immigration and let $m$ and $\lambda$ be the means of the offspring and immigration distributions, respectively. Estimation results for $m$ and $\lambda$ were obtained in the literature for the subcritical $(m < 1)$ and supercritical $(m > 1)$ cases, but no unified estimation procedure was developed, which would allow inference without knowledge of the range of $m$. The goal of this paper is to investigate this problem.
Let $\alpha_n$ be the classical empirical process and $T: D\lbrack 0, 1\rbrack \rightarrow R$. Assume $T$ satisfies the Lipschitz condition. Using the Komlos-Major-Tusnady inequality, bounds for $P(T(\alpha_n) \geq x_n \sqrt n)$ are obtained for every $n$ and $x_n > 0$. Hence expansions for large deviations, as well as some moderate and Cramer-type large-deviations results for $T(\alpha_n)$, are derived.
The Berry-Esseen-type bounds of order $N^{-1/2}$ for the rate of convergence to normality are derived for the signed linear rank statistics under the hypothesis of symmetry. The results are obtained with a broad range of regression constants and scores (allowed to be generated by discontinuous score generating functions, but not necessarily) restricted by only mild conditions, while almost all previous results are obtained with continuously differentiable score generating…
The problem of testing linear hypotheses about the parameter vector of an autoregressive process with finite order is considered. Based on the property of local asymptotic normality, we derive asymptotically optimal statistical tests. Additionally, we define and investigate so-called residual rank tests. For these tests we obtain under the null hypothesis an asymptotic distribution which does not depend on the distribution of the innovation.
Suppose $X_1, X_2, \ldots$ are independent random variables such that for some unknown $\nu$, each of $X_1, \ldots, X_{\nu - 1}$ is distributed according to $F_0$, while $X_\nu, X_{\nu + 1}, \ldots$ are all distributed according to $F_1$. We prove a result of Moustakides that claims that the CUSUM procedures are optimal in the sense of Lorden. We do that by proving that the procedures are Bayes for some stochastic mechanism of generating $\nu$.
Three types of asymptotic $\chi^2$ goodness-of-fit tests derived under the normal assumption have been used widely in factor analysis. Asymptotic behavior of the test statistics is investigated here for the factor analysis model with linearly or nonlinearly restricted factor loadings under weak assumptions on the factor vector and the error vector. In particular the limiting $\chi^2$ result for the three tests is shown to hold for the factor vector, either fixed or random…
We study the asymptotic normality of the jackknife histogram. For one sample mean, it holds if and only if $r$, the number of observations retained, and $d (= n - r)$, the number of observations deleted, both diverge to infinity. The best convergence rate $n^{-1/2}$ is obtained when $r = O(n)$ and $d = O(n)$. For $U$ statistics of degree 2 and nonlinear statistics admitting the expansion (3.1), similar results are obtained under conditions on $r$ and $d$. A second order…
David L. Donoho · Richard C. Liu · Brenda MacGibbon·刊期 1990-09-01 · 卷 18(3)·The Annals of Statistics已编入正刊
超矩形上的极小极大风险及其含义AI 解读
Consider estimating the mean of a standard Gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in $l_2$. Such sets include ellipsoids, hyperrectangles and $l_p$-bodies with $p > 2$. The minimax risk among linear estimates is within 25% of the minimax risk among all estimates. The minimax risk among truncated series estimates is within a factor 4.44 of the minimax risk. This implies that the difficulty of estimation--a statistical…
In inverse regression problems (or more generally, the estimation of ratios of regression parameters) and errors-in-variables models, it has been shown by Gleser and Hwang that the length of any confidence interval with positive confidence level is infinite with positive probability. Therefore the confidence sets derived using asymptotic theory, although having correct asymptotic coverage probability, typically have zero confidence level when the sample size is fixed. Is it…
Time-sequential point estimation is studied in the model of fully parametric censored data and Cox's regression model. Both are investigated in the context of counting processes through estimating equations defined by martingales. The concept of information and a related inequality developed in estimating function theory by Godambe are adapted to these models. These suggest some optimality criteria for choosing stopping times as well as estimators. These lead naturally to…
Observations $y_i$ are made at points $x_i$ according to the model $y_i = F(x_i) + e_i$, where the $e_i$ are independent normals with constant variance. In order to decide whether or not $F(x)$ is constant, a likelihood ratio test is constructed, comparing $F(x) \equiv \mu$ with $F(x) = \mu + Z(x)$, where $Z(x)$ is a Brownian motion. The ratio of error variance to Brownian motion variance is chosen to maximize the likelihood, and the resulting maximum likelihood statistic $B$…
Lower bounds on Bayes factors in favor of the null hypothesis in multinomial tests of point null hypotheses are developed. These are then applied to derive lower bounds on Bayes factors in both exact and asymptotic chi-squared testing situations. The general conclusion is that the lower bounds tend to be substantially larger than $P$-values, raising serious questions concerning the routine use of moderately small $P$-values (e.g., 0.05) to represent significant evidence…
Several authors have constructed nonparametric Bayes estimators for a cumulative distribution function based on (possibly right-censored) data. The prior distributions have, for example, been Dirichlet processes or, more generally, processes neutral to the right. The present article studies the related problem of finding Bayes estimators for cumulative hazard rates and related quantities, w.r.t. prior distributions that correspond to cumulative hazard rate processes with…
To check the validity of an assumed parametric model for survival data, one may compare $\hat{A}(t)$, the nonparametric Nelson-Aalen plot of the cumulative hazard rate, with $A(t, \hat{\theta})$, the estimated parametric cumulative hazard rate, $\hat{\theta}$ being for example the maximum likelihood estimator. Convergence in distribution of $\sqrt n (\hat{A}(t) - A(t, \hat{\theta}))$ and more general processes is studied in the present paper, employing the general framework…
A method for robust nonparametric regression is discussed. We consider kernel M-estimates of the regression function using Huber's ψ-function and extend results of Hardle and Gasser to the case of random designs. A practical adaptive procedure is proposed consisting of simultaneously minimising a cross-validatory criterion with respect to both the smoothing parameter and a robustness parameter occurring in the ψ-function. This method is shown to possess a…
The Asymptotics of S-Estimators in the Linear Regression Model
We consider the consistency and weak convergence of S-estimators in the linear regression model. Sufficient conditions for consistency with varying dimension are given which are sufficiently weak to cover, for example, polynomial trends and i.i.d. carriers. A weak convergence theorem for the Hampel-Rousseeuw least median of squares estimator is obtained, and it is shown under rather general conditions that the correct norming factor is n1/3. Finally, the asymptotic…
Any multiresponse estimation experiment requires a decision about the number of observations to be taken. If the covariance is unknown, no fixed-sample-size procedure can guarantee that the joint confidence region will have an assigned shape and level. Double-sampling procedures use a preliminary sample of size m to determine the minimum number of additional observations needed to achieve a prescribed accuracy and coverage probability for the parameter estimates. The…
Let {Xn} be the branching process with immigration and let m and λ be the means of the offspring and immigration distributions, respectively. Estimation results for m and λ were obtained in the literature for the subcritical (m<1) and supercritical (m>1) cases, but no unified estimation procedure was developed, which would allow inference without knowledge of the range of m. The goal of this paper is to investigate this problem.
Let αn be the classical empirical process and T:D[0,1]→R. Assume T satisfies the Lipschitz condition. Using the Komlos-Major-Tusnady inequality, bounds for P(T(αn)≥xnn) are obtained for every n and xn>0. Hence expansions for large deviations, as well as some moderate and Cramer-type large-deviations results for T(αn), are derived.
The Berry-Esseen-type bounds of order N−1/2 for the rate of convergence to normality are derived for the signed linear rank statistics under the hypothesis of symmetry. The results are obtained with a broad range of regression constants and scores (allowed to be generated by discontinuous score generating functions, but not necessarily) restricted by only mild conditions, while almost all previous results are obtained with continuously differentiable score generating…
Suppose X1,X2,… are independent random variables such that for some unknown ν, each of X1,…,Xν−1 is distributed according to F0, while Xν,Xν+1,… are all distributed according to F1. We prove a result of Moustakides that claims that the CUSUM procedures are optimal in the sense of Lorden. We do that by proving that the procedures are Bayes for some stochastic mechanism of generating ν.
Three types of asymptotic χ2 goodness-of-fit tests derived under the normal assumption have been used widely in factor analysis. Asymptotic behavior of the test statistics is investigated here for the factor analysis model with linearly or nonlinearly restricted factor loadings under weak assumptions on the factor vector and the error vector. In particular the limiting χ2 result for the three tests is shown to hold for the factor vector, either fixed or random…
We study the asymptotic normality of the jackknife histogram. For one sample mean, it holds if and only if r, the number of observations retained, and d(=n−r), the number of observations deleted, both diverge to infinity. The best convergence rate n−1/2 is obtained when r=O(n) and d=O(n). For U statistics of degree 2 and nonlinear statistics admitting the expansion (3.1), similar results are obtained under conditions on r and d. A second order…
Consider estimating the mean of a standard Gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in l2. Such sets include ellipsoids, hyperrectangles and lp-bodies with p>2. The minimax risk among linear estimates is within 25% of the minimax risk among all estimates. The minimax risk among truncated series estimates is within a factor 4.44 of the minimax risk. This implies that the difficulty of estimation--a statistical…
Observations yi are made at points xi according to the model yi=F(xi)+ei, where the ei are independent normals with constant variance. In order to decide whether or not F(x) is constant, a likelihood ratio test is constructed, comparing F(x)≡μ with F(x)=μ+Z(x), where Z(x) is a Brownian motion. The ratio of error variance to Brownian motion variance is chosen to maximize the likelihood, and the resulting maximum likelihood statistic B…
Lower bounds on Bayes factors in favor of the null hypothesis in multinomial tests of point null hypotheses are developed. These are then applied to derive lower bounds on Bayes factors in both exact and asymptotic chi-squared testing situations. The general conclusion is that the lower bounds tend to be substantially larger than P-values, raising serious questions concerning the routine use of moderately small P-values (e.g., 0.05) to represent significant evidence…
To check the validity of an assumed parametric model for survival data, one may compare A^(t), the nonparametric Nelson-Aalen plot of the cumulative hazard rate, with A(t,θ^), the estimated parametric cumulative hazard rate, θ^ being for example the maximum likelihood estimator. Convergence in distribution of n(A^(t)−A(t,θ^)) and more general processes is studied in the present paper, employing the general framework…