An asymptotic justification of the Bayesian bootstrap is given. Large-sample Bayesian bootstrap probability intervals for the mean, the variance and bands for the distribution, the smoothed density and smoothed rate function are also provided.
Differential geometric methods are used to construct approximate confidence regions for curved exponential families. The $\alpha$-connection geometries discussed by Amari (1982), and another geometry introduced here, the $c$ geometry, are exploited to construct confidence regions. Survival and case-control studies with general relative risk functions are interpreted in the context of curved exponential families, and an example illustrates the construction of confidence…
Estimates of the linear model parameters in a linear transformation model with unknown increasing transformation are obtained by maximizing a partial likelihood. A resampling scheme (likelihood sampler) is used to compute the maximum partial likelihood estimates. It is shown that for a certain "local" parameter set where the "signal to noise ratio" is small, it is asymptotically possible to estimate the linear model parameters using the partial likelihood as well as if the…
When the $\log$-likelihood statistic is divided by its mean, or an approximation to its mean, the limiting chi-squared distribution is often correct to order $n^{-3/2}$. Similarly, when the signed version of the likelihood ratio statistic is standardized with respect to its mean and variance the normal approximation is correct to order $n^{-3/2}$. Proofs for these statements have been given in great generality in the literature for the case of continuous observations. In…
Asymptotic moments of normed sample size savings are derived for one-sided curtailed nonparametric tests for symmetry for testing location shift hypotheses under the null hypothesis and contiguous alternatives. The contiguous alternative results are derived using Feller's (1943) central limit theorem result for large deviations and techniques used in Albers, Bickel and van Zwet (1976).
Consider the problem of estimating simultaneously the means $\theta_i$ of independent normal random variables $x_i$ with unit variance. Under the weighted quadratic loss $L(\theta, a) = \sum_i\lambda_i(\theta_i - a_i)^2$ with positive weights it is well known that: (1) An estimator which is admissible under one set of weights is admissible under all weights. (2) Estimating individual coordinates by proper Bayes estimators results in an admissible estimator. (3) Estimating…
Leon Jay Gleser · Raymond J. Carroll · Paul P. Gallo·刊期 1987-03-01 · 卷 15(1)·The Annals of Statistics已编入正刊
测量误差回归模型中最小二乘估计的极限分布AI 解读
It is well-known that the ordinary least squares (OLS) estimator $\hat{\beta}$ of the slope and intercept parameters $\beta$ in a linear regression model with errors of measurement for some of the independent variables (predictors) is inconsistent. However, Gallo (1982) has shown that certain linear combinations of $\beta$. In this paper, it is shown that under reasonable regularity conditions such linear combinations of $\hat{\beta}$ are (jointly) asymptotically normally…
By the term $k$-statistic or polykay, we mean an unbiased estimate of a cumulant or product of cumulants [Fisher (1929) and Tukey (1950), 1956)]. In this paper, two sets of unbiased estimates are given for the case where the mean response, $E(Y)$, depends linearly on known covariates $x$. The $k$'s are symmetric functions of the least-squares residuals and have previously been discussed by Anscombe (1961; 1981, Appendix 2). The $l$'s are optimal in the sense of having…
In the model $Y_i = g(t_i) + \varepsilon_i,\quad i = 1,\cdots, n,$ where $Y_i$ are given observations, $\varepsilon_i$ i.i.d. noise variables and $t_i$ nonrandom design points, kernel estimators for the regression function $g(t)$ with variable bandwidth (smoothing parameter) depending on $t$ are proposed. It is shown that in terms of asymptotic integrated mean squared error, kernel estimators with such a local bandwidth choice are superior to the ordinary kernel estimators…
In the setting of kernel density estimation, data-driven bandwidth, i.e., smoothing parameter, selectors are considered. It is seen that there is a well-defined, and surprisingly restrictive, bound on the rate of convergence of any automatic bandwidth selection method to the optimum. The method of least squares cross-validation achieves this bound.
In the setting of nonparametric multivariate density estimation, theorems are established which allow a comparison of the Kullback-Leibler and the least-squares cross-validation methods of smoothing parameter selection. The family of delta sequence estimators (including kernel, orthogonal series, histogram and histospline estimators) is considered. These theorems also show that either type of cross validation can be used to compare different estimators (e.g., kernel versus…
This paper is concerned with establishing a broad class of estimators (and the limiting behavior, thereof) for parametrizations of higher than second-order structure. This includes parametrizations which reflect, for example, such properties as nonlinearity and/or non-Gaussianity and/or time irreversibility. Asymptotic distributions, almost-sure convergence, and probability-one bounds for such estimators are established. Several applications of such estimators are discussed.
We consider the estimation problem for the parameter $\vartheta_0$ of a stationary ARMA $(p, q)$ process, with independent and identically, but not necessary normally distributed errors. First we prove local asymptotic normality (LAN) for this model. Then we construct locally asymptotically minimax (LAM) estimators, which asymptotically achieve the smallest possible covariance matrix. Utilizing these, we finally obtain strongly adaptive estimators, by using usual kernel…
For the analysis of nonstationary categorical time series, a parsimonious and flexible class of models is proposed. These models are generalizations of regression models for stochastically independent categorical observations. Consistency, asymptotic normality and efficiency of the maximum likelihood estimator are shown under weak and easily verifiable requirements. Some models for binary time series are discussed in detail. To demonstrate asymptotic properties, a theorem is…
A new sieve estimator for the mean function $m(t)$ of a general Gaussian process of known covariance is presented. The estimator $\hat{m}(t)$ is given explicitly from the data and has a simple distribution. It is shown that $\hat{m}(t)$ is asymptotically unbiased and consistent (weakly and in mean square) at each $t$, and that $\hat{m}$ is strongly consistent for $m$ in an appropriate norm. No assumptions are made about the "time" parameter or the covariance.
Limit distributions of prediction errors when the parameters are estimated are obtained for a general class of nonstationary processes that includes supercritical branching processes and explosive autoregressive processes as typical examples. The estimated prediction errors are used to construct new tests of fit whose limit distributions are also derived.
For the general linear model with independent errors, we propose and examine the large sample properties of an estimator of the regression parameter. In the location model, the estimator has the same properties as the trimmed mean and the robustness and efficiency properties of the trimmed mean carry over to the general model. The estimator depends on a preliminary estimate of the regression parameter and the residuals based on it. The properties of the adaptive estimator…
Let $x_1, x_2, \cdots$ be independent and normally distributed with unknown mean $\theta$ and variance 1. Let $\tau = \inf \{n \geq 1: |s_n| \geq \sqrt{2a(n + c)}\}$. Then a repeated significance test for a normal mean rejects the hypothesis $\theta = 0$ if and only if $\tau \leq N_0$ for some positive integer $N_0$. The problem we consider is estimation of $\theta$ based on the data $x_1,\cdots, x_T, T = \min\{\tau, N_0\}$. We shall solve this problem by obtaining the…
The existence and uniqueness of a limiting form of a Huber-type $M$-estimator of multivariate scatter is established under certain conditions on the observed sample. These conditions hold with probability one when sampling randomly from a continuous multivariate distribution. The existence of the estimator is proven by showing that it is the limiting point of a specific algorithm. Hence, the proof is constructive. For continuous populations, the estimator of multivariate…
This paper addresses the problem of linear regression estimation when the disturbances follow a stationary process with its spectral density known only to be in a neighborhood of some specified spectral density, for instance, that of white noise. Rather than trying to adapt to a small unspecified autocorrelation, we follow here the robustness approach, and establish the extent of the regressors and disturbance spectra interaction which require serial correlation correction.…
Suppose that $X$ has a binomial distribution $B(n, p)$, with known $p \in (0, 1)$ and unknown $n \in \{1, 2, \cdots\}$. A natural estimator for $n$ is given by $T(0) = 1, T(x) = x/p, x = 1, 2, \cdots$. This estimator is shown to be inadmissible under quadratic loss. It is shown that modifying $T(0)$ to $T(0) = -(1 - p)/(p \ln p)$ results in an admissible estimator. For $p \geq \frac{1}{2}$ it is further shown that this is the only admissible modification of $T(0)$. A partial…
The problem of finding classes of estimators which dominate the usual estimator $X$ of the mean vector $\mu$ of a $p$-variate normal distribution $(p \geq 3)$ under general quadratic loss is analytically difficult in cases where the covariance matrix is unknown. Estimators of $\mu$ in this case depend upon $X$ and an independent Wishart matrix $W$. In the present paper, integration-by-parts methods for both the multivariate normal and Wishart distributions are combined to…
In this paper Ferguson's univariate normal results for detection of outliers with variance slippage are extended to the multivariate elliptically symmetric case with dispersion slippage. The locally optimum test we derive possesses all three robustness properties, optimality, null and nonnull, and is based on Mardia's multivariate kurtosis statistic.
In order to improve the rate of decrease of the IMSE for nonparametric kernel density estimators with nonrandom bandwidth beyond $O(n^{-4/5})$ all current methods must relax the constraint that the density estimate be a bona fide function, that is, be nonnegative and integrate to one. In this paper we show how to achieve similar improvement without relaxing any of these constraints. The method can also be applied for orthogonal series, adaptive orthogonal series, spline,…
Differential geometric methods are used to construct approximate confidence regions for curved exponential families. The α-connection geometries discussed by Amari (1982), and another geometry introduced here, the c geometry, are exploited to construct confidence regions. Survival and case-control studies with general relative risk functions are interpreted in the context of curved exponential families, and an example illustrates the construction of confidence…
When the log-likelihood statistic is divided by its mean, or an approximation to its mean, the limiting chi-squared distribution is often correct to order n−3/2. Similarly, when the signed version of the likelihood ratio statistic is standardized with respect to its mean and variance the normal approximation is correct to order n−3/2. Proofs for these statements have been given in great generality in the literature for the case of continuous observations. In…
Consider the problem of estimating simultaneously the means θi of independent normal random variables xi with unit variance. Under the weighted quadratic loss L(θ,a)=∑iλi(θi−ai)2 with positive weights it is well known that: (1) An estimator which is admissible under one set of weights is admissible under all weights. (2) Estimating individual coordinates by proper Bayes estimators results in an admissible estimator. (3) Estimating…
It is well-known that the ordinary least squares (OLS) estimator β^ of the slope and intercept parameters β in a linear regression model with errors of measurement for some of the independent variables (predictors) is inconsistent. However, Gallo (1982) has shown that certain linear combinations of β. In this paper, it is shown that under reasonable regularity conditions such linear combinations of β^ are (jointly) asymptotically normally…
k-Statistics and Dispersion Effects in Regression
By the term k-statistic or polykay, we mean an unbiased estimate of a cumulant or product of cumulants [Fisher (1929) and Tukey (1950), 1956)]. In this paper, two sets of unbiased estimates are given for the case where the mean response, E(Y), depends linearly on known covariates x. The k's are symmetric functions of the least-squares residuals and have previously been discussed by Anscombe (1961; 1981, Appendix 2). The l's are optimal in the sense of having…
In the model Yi=g(ti)+εi,i=1,⋯,n, where Yi are given observations, εi i.i.d. noise variables and ti nonrandom design points, kernel estimators for the regression function g(t) with variable bandwidth (smoothing parameter) depending on t are proposed. It is shown that in terms of asymptotic integrated mean squared error, kernel estimators with such a local bandwidth choice are superior to the ordinary kernel estimators…
We consider the estimation problem for the parameter ϑ0 of a stationary ARMA (p,q) process, with independent and identically, but not necessary normally distributed errors. First we prove local asymptotic normality (LAN) for this model. Then we construct locally asymptotically minimax (LAM) estimators, which asymptotically achieve the smallest possible covariance matrix. Utilizing these, we finally obtain strongly adaptive estimators, by using usual kernel…
A new sieve estimator for the mean function m(t) of a general Gaussian process of known covariance is presented. The estimator m^(t) is given explicitly from the data and has a simple distribution. It is shown that m^(t) is asymptotically unbiased and consistent (weakly and in mean square) at each t, and that m^ is strongly consistent for m in an appropriate norm. No assumptions are made about the "time" parameter or the covariance.
Let x1,x2,⋯ be independent and normally distributed with unknown mean θ and variance 1. Let τ=inf{n≥1:∣sn∣≥2a(n+c)}. Then a repeated significance test for a normal mean rejects the hypothesis θ=0 if and only if τ≤N0 for some positive integer N0. The problem we consider is estimation of θ based on the data x1,⋯,xT,T=min{τ,N0}. We shall solve this problem by obtaining the…
A Distribution-Free M-Estimator of Multivariate Scatter
The existence and uniqueness of a limiting form of a Huber-type M-estimator of multivariate scatter is established under certain conditions on the observed sample. These conditions hold with probability one when sampling randomly from a continuous multivariate distribution. The existence of the estimator is proven by showing that it is the limiting point of a specific algorithm. Hence, the proof is constructive. For continuous populations, the estimator of multivariate…
Suppose that X has a binomial distribution B(n,p), with known p∈(0,1) and unknown n∈{1,2,⋯}. A natural estimator for n is given by T(0)=1,T(x)=x/p,x=1,2,⋯. This estimator is shown to be inadmissible under quadratic loss. It is shown that modifying T(0) to T(0)=−(1−p)/(plnp) results in an admissible estimator. For p≥21 it is further shown that this is the only admissible modification of T(0). A partial…
The problem of finding classes of estimators which dominate the usual estimator X of the mean vector μ of a p-variate normal distribution (p≥3) under general quadratic loss is analytically difficult in cases where the covariance matrix is unknown. Estimators of μ in this case depend upon X and an independent Wishart matrix W. In the present paper, integration-by-parts methods for both the multivariate normal and Wishart distributions are combined to…
In order to improve the rate of decrease of the IMSE for nonparametric kernel density estimators with nonrandom bandwidth beyond O(n−4/5) all current methods must relax the constraint that the density estimate be a bona fide function, that is, be nonnegative and integrate to one. In this paper we show how to achieve similar improvement without relaxing any of these constraints. The method can also be applied for orthogonal series, adaptive orthogonal series, spline,…