A general missing information principle is proposed for constructing $M$-estimators of regression parameters in the presence of left truncation and right censoring on the observed responses. By making use of martingale central limit theorems and empirical process theory, the asymptotic normality of $M$-estimators is established under certain assumptions. Asymptotically efficient $M$-estimators are also developed by using data-dependent score functions. In addition,…
Formal posteriors for improper priors are investigated in connection with coherence, both in the sense of Regazzini and of Heath and Sudderth. Those priors $\pi$ which are linked with the improper prior $\gamma$ by the relation $\pi(B) = 0$ whenever $\gamma(B) < +\infty$ are studied in particular. Moreover, a characterization of the inferences which are coherent according to Heath and Sudderth is found, and several examples, exhibiting several phenomena, are given.
Extending work of Rubin, this paper explores a Bayesian counterpart of the classical $p$-value, namely, a tail-area probability of a "test statistic" under a null hypothesis. The Bayesian formulation, using posterior predictive replications of the data, allows a "test statistic" to depend on both data and unknown (nuisance) parameters and thus permits a direct measure of the discrepancy between sample and population quantities. The tail-area probability for a "test…
Jayanta K. Ghosh · Pranab K. Sen · Rahul Mukerjee·刊期 1994-09-01 · 卷 22（3）·The Annals of Statistics已编入正刊
二阶 Pitman 接近性与 Pitman 可容许性AI 解读
Motivated by the first-order Pitman closeness of best asymptotically normal estimators and some recent developments on higher-order asymptotic efficiency of estimators, a second-order asymptotic theory is developed for comparison of estimators under the Pitman closeness criterion, covering both the cases without and with nuisance parameters. The notion of second-order Pitman admissibility is also developed.
Luc Devroye · Laszlo Gyorfi · Adam Krzyzak · Gabor Lugosi·刊期 1994-09-01 · 卷 22（3）·The Annals of Statistics已编入正刊
最近邻回归函数估计的强普适一致性AI 解读
Two results are presented concerning the consistency of the $k$-nearest neighbor regression estimate. We show that all modes of convergence in $L_1$ (in probability, almost sure, complete) are equivalent if the regression variable is bounded. Under the additional conditional $k/\log n \rightarrow \infty$ we also obtain the strong universal consistency of the estimate.
Nonparametric regression using locally weighted least squares was first discussed by Stone and by Cleveland. Recently, it was shown by Fan and by Fan and Gijbels that the local linear kernel-weighted least squares regression estimator has asymptotic properties making it superior, in certain senses, to the Nadaraya-Watson and Gasser-Muller kernel estimators. In this paper we extend their results on asymptotic bias and variance to the case of multivariate predictor variables.…
Estimation of a covariance matrix $\sum$ is a notoriously difficult problem; the standard unbiased estimator can be substantially suboptimal. We approach the problem from a noninformative prior Bayesian perspective, developing the reference noninformative prior for a covariance matrix and obtaining expressions for the resulting Bayes estimators. These expressions involve the computation of high-dimensional posterior expectations, which is done using a recent Markov chain…
Under the usual assumptions of normality, the recursive estimator known as the Kalman filter gives excellent results and has found an extremely broad field of application--not only for estimating the state of a stochastic dynamic system, but also for estimating model parameters as well as detecting abrupt changes in the states or the parameters. It is well known, however, that significantly nonnormal noise, and particularly the presence of outliers, severely degrades the…
This article considers a case of parametric bootstrap when the observations consist of generation sizes of the branching process with immigration together with the immigration component of each generation. Suppose we estimate the offspring mean $m$ by the maximum likelihood estimator (m.l.e.). It is then shown that the bootstrap version of the standardized m.l.e. does not have the same limiting distribution as the standardized m.l.e., under the assumption that $m = 1$ (critic…
We apply the bootstrap for general stationary observations, proposed by Kunsch, to the empirical process for $p$-dimensional random vectors. It is known that the empirical process in the multivariate case converges weakly to a certain Gaussian process. We show that the bootstrapped empirical process converges weakly to the same Gaussian process almost surely, assuming that the block length $l$ for constructing bootstrap replicates satisfies $l(n) = O(n^{1/2-\varepsilon}), 0 < \varepsilon < \frac{1}{2}$…
We show that the empirical process of the block-based bootstrap observations from a stationary sequence converges weakly to an appropriate Gaussian process, conditionally in probability and almost surely, depending upon the block length. This bootstrap was introduced by Kunsch and later by Liu and Singh. Applications in estimation of the sampling distribution of a compactly differentiable functional are indicated.
In this paper, two versions of the Berry-Esseen theorems are established for $L$-statistics in the non-identically distributed case. One theorem, which requires $E|X_i|^3 < \infty$, is an extension of the classical Berry-Esseen theorem. Another, proved under the condition $E|X_i|^\alpha < \infty$ for some $\alpha \in (0, 1\rbrack$, seems to be of more interest for statistical inference. Some applications are also discussed.
We study linear combinations of order statistics ($L$-statistics) in survey problems with a stratified multistage sampling design. Two general types of $L$-statistics are considered: smooth $L$-statistics with weights generated by a smooth function and nonsmooth $L$-statistics (sample quantiles). The trimmed sample mean, the decile mean and variance, the sample Lorenz curve and the sample Gini's family parameters are examples of smooth $L$-statistics or functions of smooth $L$…
Based on a duality between $E$-optimality for (sub-) parameters in weighted polynomial regression and a nonlinear approximation problem of Chebyshev type, in many cases the optimal approximate designs on nonnegative and nonpositive experimental regions $\lbrack a, b\rbrack$ are found to be supported by the extrema of the only equioscillating weighted polynomial over this region with leading coefficient 1. A similar result is stated for regression on symmetric regions $\lbrack -b, b\rbrack$…
In this paper we consider experimental situations where treatments from a mixed level factorial design are to be applied to experimental units over space or time and where there may be an unknown trend effect which can be approximated by some polynomial function of the order in which the observations are obtained. A method of constructing run orders of treatments for such situations is given which generally yields least squares estimators for main effects that have a higher…
In the polynomial regression model of degree $m \in \mathbb{N}$ we consider the problem of determining a design for the identification of the correct degree of the underlying regression. We propose a new optimality criterion which minimizes a weighted $p$-mean of the variances of the least squares estimators for the coefficients of $x^l$ in the polynomial regression models of degree $l = 1,\cdots, m$. The theory of canonical moments is used to determine the optimal designs…
We use the probabilistic method to show that if $f_{nh}$ is the standard kernel estimate with smoothing factor $h$, then there exists a deterministic sequence $h_n$ such that, for all densities, $\operatornamewithlimits{\lim\inf}_{n\rightarrow\infty} \frac{\mathbf{E} \int |f_{nh_n} - f|}{\inf_h \mathbf{E} \int |f_{nh} - f|} = 1.$
This paper deals with the following approach for estimating the mean $\mu$ of an $n$-dimensional random vector $Y$: first, a family $\mathbf{S}$ of $n \times n$ matrices is specified. Then, an element $\widehat{S} \in \mathbf{S}$ is selected by Mallows $C_L$, and $\widehat{\mu} = \widehat{S}\cdot Y$. The case is considered that $\mathbf{S}$ is an "ordered linear smoother" according to some easily interpretable, qualitative conditions. Examples include linear smoothing…
Observations $X_i$ are uncorrelated with means $\theta_i, i = 1,\cdots, n$, and variances 1. The linear estimators $\widehat{\theta} = T\mathbf{X}$, for some $n \times n$ matrix $T$, are widely used in smoothing problems, where it is assumed that neighbouring parameter values are similar. The smoothness assumption is violated in change point problems, where neighbouring parameter values are equal, except at some unspecified change points where there are jumps of unknown size…
In this paper, a dynamic sampling plan in the Shiryayev-Roberts procedure is considered. It is shown that a two-rate dynamic sampling plan is optimal in the sense that it minimizes the stationary average delay time (SADT). Analytical results as well as numerical comparisons show that it is significantly superior to the fixed sampling plan. The comparison also shows that it is as powerful as the dynamic sampling procedure of Assaf and Ritov. The generalizations to the fast…
Suppose that a system in its standard state produces i.i.d. observations whose distribution is symmetric about zero. At an unknown time the system may leave its standard state, and the observations would subsequently be stochastically larger. Subject to a bound on the rate of false alarms, one wants to detect quickly such a departure from the standard state. We present a robust method of detection which is computationally feasible and remarkably efficient. The method is…
We study two modified synthetic data least-squares estimation methods for linear regression models with right censored response variables, unspecified residual distributions and random censoring variables which may not be i.i.d. These methods are the result of an investigation into the use of stratification. We conclude that stratification should be used whether or not the censoring variables are dependent on the covariates. We give the asymptotic results of the estimators…
The frailty model is a generalization of Cox's proportional hazards model which includes a random effect. Nielsen, Gill, Andersen and Sorensen proposed an EM algorithm to estimate the cumulative baseline hazard and the variance of the random effect. Here existence and consistency of the estimators are proved. An example using truncated and censored data is considered.
In this article, a model of single variable sampling plan with type I censoring is studied. Under the assumption that the variable is exponentially distributed and the loss function is a polynomial, an explicit expression of the Bayes risk is derived. Then, a simple and finite algorithm to determine an optimal sampling plan for minimizing the Bayes risk is suggested. Furthermore, a discretization method is proposed so that one can easily obtain an approximately optimal…
A Missing Information Principle and M-Estimators in Regression Analysis with Censored and Truncated Data
A general missing information principle is proposed for constructing M-estimators of regression parameters in the presence of left truncation and right censoring on the observed responses. By making use of martingale central limit theorems and empirical process theory, the asymptotic normality of M-estimators is established under certain assumptions. Asymptotically efficient M-estimators are also developed by using data-dependent score functions. In addition,…
Formal posteriors for improper priors are investigated in connection with coherence, both in the sense of Regazzini and of Heath and Sudderth. Those priors π which are linked with the improper prior γ by the relation π(B)=0 whenever γ(B)<+∞ are studied in particular. Moreover, a characterization of the inferences which are coherent according to Heath and Sudderth is found, and several examples, exhibiting several phenomena, are given.
Extending work of Rubin, this paper explores a Bayesian counterpart of the classical p-value, namely, a tail-area probability of a "test statistic" under a null hypothesis. The Bayesian formulation, using posterior predictive replications of the data, allows a "test statistic" to depend on both data and unknown (nuisance) parameters and thus permits a direct measure of the discrepancy between sample and population quantities. The tail-area probability for a "test…
Two results are presented concerning the consistency of the k-nearest neighbor regression estimate. We show that all modes of convergence in L1 (in probability, almost sure, complete) are equivalent if the regression variable is bounded. Under the additional conditional k/logn→∞ we also obtain the strong universal consistency of the estimate.
Estimation of a covariance matrix ∑ is a notoriously difficult problem; the standard unbiased estimator can be substantially suboptimal. We approach the problem from a noninformative prior Bayesian perspective, developing the reference noninformative prior for a covariance matrix and obtaining expressions for the resulting Bayes estimators. These expressions involve the computation of high-dimensional posterior expectations, which is done using a recent Markov chain…
This article considers a case of parametric bootstrap when the observations consist of generation sizes of the branching process with immigration together with the immigration component of each generation. Suppose we estimate the offspring mean m by the maximum likelihood estimator (m.l.e.). It is then shown that the bootstrap version of the standardized m.l.e. does not have the same limiting distribution as the standardized m.l.e., under the assumption that m=1 (critic…
We apply the bootstrap for general stationary observations, proposed by Kunsch, to the empirical process for p-dimensional random vectors. It is known that the empirical process in the multivariate case converges weakly to a certain Gaussian process. We show that the bootstrapped empirical process converges weakly to the same Gaussian process almost surely, assuming that the block length l for constructing bootstrap replicates satisfies l(n)=O(n1/2−ε),0<ε<21…
On the Berry-Esseen Bound for L-Statistics in the Non-I.D. Case with Applications to the Estimation of Location Parameters
In this paper, two versions of the Berry-Esseen theorems are established for L-statistics in the non-identically distributed case. One theorem, which requires E∣Xi∣3<∞, is an extension of the classical Berry-Esseen theorem. Another, proved under the condition E∣Xi∣α<∞ for some α∈(0,1], seems to be of more interest for statistical inference. Some applications are also discussed.
We study linear combinations of order statistics (L-statistics) in survey problems with a stratified multistage sampling design. Two general types of L-statistics are considered: smooth L-statistics with weights generated by a smooth function and nonsmooth L-statistics (sample quantiles). The trimmed sample mean, the decile mean and variance, the sample Lorenz curve and the sample Gini's family parameters are examples of smooth L-statistics or functions of smooth L…
E-Optimal Designs in Weighted Polynomial Regression
Based on a duality between E-optimality for (sub-) parameters in weighted polynomial regression and a nonlinear approximation problem of Chebyshev type, in many cases the optimal approximate designs on nonnegative and nonpositive experimental regions [a,b] are found to be supported by the extrema of the only equioscillating weighted polynomial over this region with leading coefficient 1. A similar result is stated for regression on symmetric regions [−b,b]…
In the polynomial regression model of degree m∈N we consider the problem of determining a design for the identification of the correct degree of the underlying regression. We propose a new optimality criterion which minimizes a weighted p-mean of the variances of the least squares estimators for the coefficients of xl in the polynomial regression models of degree l=1,⋯,m. The theory of canonical moments is used to determine the optimal designs…
We use the probabilistic method to show that if fnh is the standard kernel estimate with smoothing factor h, then there exists a deterministic sequence hn such that, for all densities, liminfn→∞infhE∫∣fnh−f∣E∫∣fnhn−f∣=1.
This paper deals with the following approach for estimating the mean μ of an n-dimensional random vector Y: first, a family S of n×n matrices is specified. Then, an element S∈S is selected by Mallows CL, and μ=S⋅Y. The case is considered that S is an "ordered linear smoother" according to some easily interpretable, qualitative conditions. Examples include linear smoothing…
Observations Xi are uncorrelated with means θi,i=1,⋯,n, and variances 1. The linear estimators θ=TX, for some n×n matrix T, are widely used in smoothing problems, where it is assumed that neighbouring parameter values are similar. The smoothness assumption is violated in change point problems, where neighbouring parameter values are equal, except at some unspecified change points where there are jumps of unknown size…