A general notion of a bootstrapped mean constructed by exchangeably weighting sample points is introduced. Consistency of this generalized bootstrapped mean, which includes proposals of Efron and Rubin among others, is proved by classical linear rank statistics theory. The consistency of generalized bootstrapped empirical and quantile processes is also established.
Techniques are developed for bootstrap estimation of conditional distributions, with application to confidence intervals and hypothesis tests for one parameter, conditional on the value of an estimator of another. Both Monte Carlo and saddlepoint methods for approximating bootstrap distributions are considered, and empirical methods are suggested for implementing these techniques. For example, in the case of Monte Carlo methods, we suggest empirical techniques for selecting…
Inference, including variance estimation, can be made from stratified samples by selecting a balanced set of subsamples. This balanced subsampling method is generically called the balanced repeated replication method in survey data analysis, which includes McCarthy's balanced half-samples method and its extensions for more general stratified designs. We establish the asymptotic consistency of the balanced repeated replication variance estimators when the parameter of…
Consider a multiple linear regression model $Y_i = x'_i\beta + \varepsilon_i$, where the $\varepsilon_i$'s are independent random variables with common distribution $F$ and the $x_i$'s are known design vectors. Let $\bar\beta_n$ be the $M$-estimator of $\beta$ corresponding to a score function $\psi$. Under some conditions on $F, \psi$ and the $x_i$'s, two-term Edgeworth expansions for the distributions of standardized and studentized $\bar\beta_n$ are obtained. Furthermore,…
The metrically trimmed mean is defined as the average of observations remaining after a fixed number of outlying observations have been removed. A metric, the distance from the median, is used to determine which points are outlying. The influence curve and the asymptotic normality of the metrically trimmed mean are derived using von Mises expansions. The relative merits of the median, the trimmed mean and the metrically trimmed mean are discussed in neighborhoods of…
We consider $M$-estimators defined by minimization of a convex criterion function, not necessarily smooth. Our asymptotic results generalize some of those concerning the LAD estimators. We establish a Bahadur-type strong approximation and bounds on the rate of convergence.
The usual confidence interval, based on Student's $t$ distribution, has conditional confidence that is larger than the nominal confidence level. Although this fact is known, along with the fact that increased conditional confidence can be used to improve a confidence assertion, the confidence assertion of Student's $t$ interval has never been critically examined. We do so here, and construct a confidence estimator that allows uniformly higher confidence in the interval and…
Bounds on the asymptotic relative efficiency (ARE) of the Box-Cox transformed two-sample $t$-test to the ordinary $t$-test are obtained under local alternatives. It is shown that the ARE is at least 1 for location-shift models. In the case of scale-shift models, a similar bound applies provided the limiting value of the estimated power transformation is greater than 1. If instead the Box-Cox transformed $t$-test is compared against the ordinary $t$-test applied to the…
Simultaneous estimation of the vector of variance components under unbalanced mixed models w.r.t. the ordinary quadratic loss function is considered. A new method of constructing invariant quadratic admissible estimators, both with and without the condition of unbiasedness, is presented. Using this method, admissible estimators for the factorial models with imbalance at the last stage and the unbalanced $(p - 1)$-way nested factors design are constructed.
A sequential procedure for estimating the regression parameter $\beta \in R^k$ in a regression model with symmetric errors is proposed. This procedure is shown to have asymptotically smaller regret than the procedure analyzed by Martinsek when $\mathbf{\beta} = \mathbf{0}$, and the same asymptotic regret as that procedure when $\mathbf{\beta} \neq \mathbf{0}$. Consequently, even when the errors are normally distributed, it follows that the asymptotic regret can be negative…
In this paper, we suggest tests of stationarity in the mean of autoregressive time series versus arbitrary trend alternatives. As an intermediate, though essential, step local asymptotic normality of autoregressive models with a nonparametric regression trend is established. Moreover, a functional central limit theorem for the underlying likelihood ratio processes is derived. These results then offer a general construction principle by which every goodness of fit test (case…
A new test is derived for the hypothesis that a regression function has a prescribed parametric form. Unlike many recent proposals, this test does not depend on arbitrarily chosen smoothing parameters. In fact, the test statistic is itself a smoothing parameter which is selected to minimize an estimated risk function. The exact distribution of the test statistic is obtained when the error terms in the regression model are Gaussian, while the large sample distribution is…
Many methods have been proposed for modelling nonhomogeneous Poisson processes, including change point models and log-linear models. In this paper, we use likelihood ratio tests to choose which of these models are necessary. Of particular interest is the test for the presence of a change point, for which standard asymptotic theory is not valid. Large deviation methods are applied to approximate the significance level, and power approximations are given. Confidence regions…
A Hellinger-type distance for hazard rate functions is defined. It is used to obtain a class of minimum distance estimators for data that are subject to a possible right censorship. The corresponding score process is shown to be approximated by a martingale, which is exploited to obtain the asymptotic normality under considerably weaker conditions than those normally assumed for minimum Hellinger distance estimators. It is also shown that under the parametric assumption the…
Adaptive nonparametric kernel estimators for the location of a peak of the spectral density of a stationary time series are proposed and investigated. They are based on direct smoothing of the periodogram where the amount of smoothing is determined automatically in an asymptotically optimal fashion. These adaptive estimators minimize the asymptotic mean squared error. Adaptivity is derived from the weak convergence of a two-parameter stochastic process in a deviation and a…
Consider the problem of estimating the value of a functional $\Lambda(f)$ for $f$ an unknown density or regression function. The straightforward plug-in estimator $\Lambda(\hat f)$ with $\hat f$ a particular estimate of $f$ achieves the optimal rate of convergence in the sense of Stone over bounded subsets of a Sobolev space for a broad class of linear and nonlinear functionals. For many functionals the rate calculation depends on a Frechet-like derivative of the functional,…
The paper is concerned with data representing a sample of smooth curves which can be considered as independent realizations of an underlying biological (chemical, $\ldots$) process. Such samples of curves often possess the following features: There is a typical structural pattern common to all curves of the sample. On the other hand, individual realizations of the typical shape show different dynamics and intensity. In particular, typical peaks are shifted from individual to…
We investigate some of the possibilities for improvement of univariate and multivariate kernel density estimates by varying the window over the domain of estimation, pointwise and globally. Two general approaches are to vary the window width by the point of estimation and by point of the sample observation. The first possibility is shown to be of little efficacy in one variable. In particular, nearest-neighbor estimators in all versions perform poorly in one and two…
Three notions of depth for directional data, angular simplicial depth (ASD), angular Tukey's depth (ATD) and arc distance depth (ADD), are developed and studied. The empirical versions of these depths give rise to center-outward rankings of angular data which may be regarded as extensions of the usual center-outward ranking on the line. Three medians derived from these depths are examined and compared. Applications in nonparametric classification and in implementing the…
A maximum-penalized-likelihood method is proposed for estimating a mixing distribution and it is shown that this method produces a consistent estimator, in the sense of weak convergence. In particular, a new proof of the consistency of maximum-likelihood estimators is given. The estimated number of components is shown to be at least as large as the true number, for large samples. Also, the large-sample limits of estimators which are constrained to have a fixed finite number…
Biological cell membranes contain molecular structures known as ion channels. The electrical activity of these channels regulates a number of cellular functions, such as heartbeat and neural transmission. Often the experimenter can only observe the combined current conducted through a cell patch containing an unknown number of ion channels, each channel of equal conductance. The observed current record is a step function with unit up or down steps occurring at random times.…
Given a parametric model and an improper prior distribution, the formal posterior distribution induces decision rules in any decision problem. The results here provide conditions under which this formal Bayes method produces admissible decision rules for all quadratically regular decision problems. The conditions derived are shown to be equivalent to the recurrence of a natural symmetric Markov chain (on the parameter space) generated by the model and the improper prior. The…
Several authors have shown how to efficiently estimate $\beta$ in the semiparametric additive model $y = x'\beta + g(t) + \text{error}$, $g(t)$ smooth but unknown when the error distribution is normal. However, the general theory suggests that efficient estimation should be possible for general error distributions with finite Fisher information even when the error distribution is unknown. In this note we construct a sequence of estimators which achieves this goal under…
A basic interpretation is given which provides a new way of understanding the structure of the species problem and which leads to the popular Turing-Good-Robbins estimator. Through this interpretation we provide an explanation why the Turing-Good-Robbins estimators are always biased. An iterative procedure is suggested and applied to these estimators, which leads to new estimators whose biases are reduced. Using this basic construction we are able to generalize our…
Bootstrapping M-Estimators of a Multiple Linear Regression Parameter
Consider a multiple linear regression model Yi=xi′β+εi, where the εi's are independent random variables with common distribution F and the xi's are known design vectors. Let βˉn be the M-estimator of β corresponding to a score function ψ. Under some conditions on F,ψ and the xi's, two-term Edgeworth expansions for the distributions of standardized and studentized βˉn are obtained. Furthermore,…
Asymptotics for M-Estimators Defined by Convex Minimization
We consider M-estimators defined by minimization of a convex criterion function, not necessarily smooth. Our asymptotic results generalize some of those concerning the LAD estimators. We establish a Bahadur-type strong approximation and bounds on the rate of convergence.
The usual confidence interval, based on Student's t distribution, has conditional confidence that is larger than the nominal confidence level. Although this fact is known, along with the fact that increased conditional confidence can be used to improve a confidence assertion, the confidence assertion of Student's t interval has never been critically examined. We do so here, and construct a confidence estimator that allows uniformly higher confidence in the interval and…
Bounds on the asymptotic relative efficiency (ARE) of the Box-Cox transformed two-sample t-test to the ordinary t-test are obtained under local alternatives. It is shown that the ARE is at least 1 for location-shift models. In the case of scale-shift models, a similar bound applies provided the limiting value of the estimated power transformation is greater than 1. If instead the Box-Cox transformed t-test is compared against the ordinary t-test applied to the…
Simultaneous estimation of the vector of variance components under unbalanced mixed models w.r.t. the ordinary quadratic loss function is considered. A new method of constructing invariant quadratic admissible estimators, both with and without the condition of unbiasedness, is presented. Using this method, admissible estimators for the factorial models with imbalance at the last stage and the unbalanced (p−1)-way nested factors design are constructed.
A sequential procedure for estimating the regression parameter β∈Rk in a regression model with symmetric errors is proposed. This procedure is shown to have asymptotically smaller regret than the procedure analyzed by Martinsek when β=0, and the same asymptotic regret as that procedure when β=0. Consequently, even when the errors are normally distributed, it follows that the asymptotic regret can be negative…
Consider the problem of estimating the value of a functional Λ(f) for f an unknown density or regression function. The straightforward plug-in estimator Λ(f^) with f^ a particular estimate of f achieves the optimal rate of convergence in the sense of Stone over bounded subsets of a Sobolev space for a broad class of linear and nonlinear functionals. For many functionals the rate calculation depends on a Frechet-like derivative of the functional,…
The paper is concerned with data representing a sample of smooth curves which can be considered as independent realizations of an underlying biological (chemical, …) process. Such samples of curves often possess the following features: There is a typical structural pattern common to all curves of the sample. On the other hand, individual realizations of the typical shape show different dynamics and intensity. In particular, typical peaks are shifted from individual to…
Several authors have shown how to efficiently estimate β in the semiparametric additive model y=x′β+g(t)+error, g(t) smooth but unknown when the error distribution is normal. However, the general theory suggests that efficient estimation should be possible for general error distributions with finite Fisher information even when the error distribution is unknown. In this note we construct a sequence of estimators which achieves this goal under…