In a general linear model, $Y = X\beta + R$ with $Y$ and $R n$-dimensional, $X$ a $n \times p$ matrix, and $\beta p$-dimensional, let $\hat\beta$ be an $M$ estimator of $\beta$ satisfying $0 = \sum x_i\psi(y_i - x'_i\beta)$. Let $p \rightarrow \infty$ such that $(p \log n)^{3/2} /n \rightarrow 0$. Then $\max_i|x'_i(\hat{\beta} - \beta)| \rightarrow _P 0$, and it is possible to find a uniform normal approximation for the distribution of $\hat{\beta}$ under which arbitrary…
The partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood (GML) estimate for the smoothing parameter. Then this estimate is compared with the generalized cross validation (GCV) estimate both analytically and by Monte Carlo methods. The comparison is based on a predictive mean square error criteria. It is shown that if the true, unknown function being estimated is smooth in a sense to be defined then the GML…
This paper concerns the method of generalized cross validation (GCV), a promising way of choosing between linear estimates. Based on Stein estimates and the associated unbiased risk estimates (Stein, 1981), a new approach to GCV is developed. Many consistency results are obtained for the cross-validated (Steinized) estimates in the contexts of nearest-neighbor nonparametric regression, model selection, ridge regression, and smoothing splines. Moreover, the associated Stein's…
In a logistic regression model when covariates are subject to measurement error the naive estimator, obtained by regressing on the observed covariates, is asymptotically biased. We introduce a bias-adjusted estimator and two estimators appropriate for normally distributed measurement errors -a functional maximum likelihood estimator and an estimator which exploits the consequences of sufficiency. The four proposals are studied asymptotically under conditions which are…
Self-consistent estimators for estimating distribution functions from incomplete data are presented. In many cases these estimators are also generalized maximum likelihood estimators. In this paper we discuss the theoretical properties of such estimators: existence, uniform consistency, law of the iterated logarithm, and weak convergence. Applications to the product limit estimator for right-censored data and to the estimator proposed by Turnbull (1974, 1976) for doubly…
The likelihood is defined for a state space model with incompletely specified initial conditions by transforming the data to eliminate the dependence on the unspecified conditions. This approach is extended to obtain estimates of the state vectors and predictors and interpolators for missing observations. It is then shown that this method is equivalent to placing a diffuse prior distribution on the unspecified part of the initial state vector, and modified versions of the…
Probability forecasts for a sequence of uncertain events may be compared with the outcomes of those events by means of a natural criterion of empirical validity, calibration. It is shown that any two sequences of forecasts which both meet this criterion must be in asymptotic agreement. These agreed values can then be considered as correct objective probability forecasts for the particular sequence of outcome results obtained. However, the objective forecasts vary with the…
For fixed censoring models that contain at most one intermediate censoring point, we obtain exact algebraic expressions for the asymptotic variances of (i) the quantiles of the Kaplan-Meier (KM, 1958) survival estimator and (ii) the KM estimator itself at fixed time points. The relationship between (i) and (ii) is found to be the same as the one derived by Sander (1975) and Reid (1981b) for the random censorship model. Confidence intervals for the quantiles based on (i) are…
P. Janssen · J. Jureckova · N. Veraverbeke·刊期 1985-09-01 · 卷 13（3）·The Annals of Statistics已编入正刊
一步与两步 M 估计量的收敛速度：最大似然与 Pitman 估计的应用AI 解读
A one-step version $M^{(1)}_n$ and a two-step version $M^{(2)}_n$ of a general $M$-estimator $M_n$ are suggested such that $M_n - M^{(1)}_n = O_p(n^{-1})$ and $M_n - M^{(2)}_n = O_p(n^{-3/2})$ for every $n^{1/2}$-consistent initial estimator and under some regularity conditions. In the special case of maximum likelihood estimation, this among other yields that the second-order efficiency properties of $M^{(2)}_n$ coincide with those of $M_n$. An application to the Pitman…
This paper examines how a Bayesian decision maker would update his/her probability $p$ for the occurrence of an event $A$ in the light of a number of expert opinions expressed as probabilities $q_1, \cdots, q_n$ of $A$. It is seen, among other things, that the linear opinion pool, $\lambda_0p + \sum^n_{i = 1} \lambda_iq_i$, corresponds to an application of Bayes' Theorem when the decision maker has specified only the mean of the marginal distribution for $(q_1, \cdots, q_n)$…
Marc Hallin · Jean-Francois Ingenbleek · Madan L. Puri·刊期 1985-09-01 · 卷 13（3）·The Annals of Statistics已编入正刊
针对 ARMA 备择的随机性线性序列秩检验AI 解读
In this paper we introduce a class of linear serial rank statistics for the problem of testing white noise against alternatives of ARMA serial dependence. The asymptotic normality of the proposed statistics is established, both under the null as well as alternative hypotheses, using LeCam's notion of contiguity. The efficiency properties of the proposed statistics are investigated, and an explicit formulation of the asymptotically most efficient score-generating functions is…
Two families of invariant tests for independence of random variables on compact Riemannian manifolds are proposed and studied. The tests are based on Gine's Sobolev norms which are obtained by mapping the manifolds into Hilbert spaces. For general compact manifolds, randomization tests are suggested. For the bivariate circular case, distribution-free tests based on uniform scores are considered.
We may take observations sequentially from a population with unknown mean $\theta$. After this sampling stage, we are to decide whether $\theta$ is greater or less than a known constant $\nu$. The net worth upon stopping is either $\theta$ or $\nu$, respectively, minus sampling costs. The objective is to maximize the expected net worth when the probability measure of the observations is a Dirichlet process with parameter $\alpha$. The stopping problem is shown to be…
The problem is to estimate a quantile of a symmetric distribution. The cases of known and unknown center are studied for small and large samples. The estimators for known center are the sample quantile, the symmetrized sample quantile, the sample quantile from flipped over data, the Rao-Blackwellized sample quantile, and a Bayes estimator using a Dirichlet prior. For center unknown, we study the analogues of the first four estimators listed above. For small samples and…
Suppose that $F_0$ is a population which is symmetric about zero, so that $F(\cdot) = F_0(\cdot - \theta)$ is symmetric about $\theta$. We consider the problem of estimating $F_0$ (shape parameter), both $\theta$ and $F_0$, and $F$ based on a random sample from $F$. First, some asymptotically minimax bounds are obtained. Then, some estimates are constructed which are asymptotically minimax-efficient (the risks of which achieve the minimax bounds uniformly). Furthermore, it…
In the framework of Huber's theory of robust estimation of a location parameter, minimax variance M-estimators are studied for error distributions with densities of the form $f(x) = (1 - \varepsilon)h(x) + \varepsilon g(x)$, where $g$ is unknown. A well-known result of Huber (1964) is that when $h$ is strongly unimodal, the least informative density $f_0 = (1 - \varepsilon)h + \varepsilon g_0$ has exponential tails. We study the minimax variance solutions when the known…
Sandor Csorgo · Paul Deheuvels · David Mason·刊期 1985-09-01 · 卷 13（3）·The Annals of Statistics已编入正刊
分布尾指数的核估计AI 解读
We introduce a new estimate of the exponent of a distribution whose tail varies regularly at infinity. This estimate is expressed as the convolution of a kernel with the logarithm of the quantile function, and includes as particular cases the estimates introduced by Hill and by De Haan. Under very weak conditions, we prove asymptotic normality, consistency and discuss the optimal choices of the kernel and of the bandwidth parameter.
A sample $X_1, \cdots, X_n$ of i.i.d. $R^d$-valued random vectors with common density $f$ is used to construct the density estimate $f_n(x) = (1/n) \sum^n_{i = 1} H^{-d}_{ni}K((x - X_i)/H_{ni}),$ where $K$ is a given density on $R^d$, and the $H_{ni}$'s are positive functions of $n, i$ and $X_1, \cdots, X_n$ (but not of $x$). The $H_{ni}$'s can be thought of as locally adapted smoothing parameters. We give sufficient conditions for the weak convergence to 0 of $\int |f_n - f|$…
A data-driven method of choosing the bandwidth, $h$, of a kernel density estimator is heuristically motivated by considering modifications of the Kullback-Leibler or pseudo-likelihood cross-validation function. It is seen that this means of choosing $h$ is asymptotically equivalent to taking the $h$ that minimizes some compelling error criteria such as the average squared error and the integrated squared error. Thus, for a given kernel function, the bandwidth can be chosen…
For nonparametric regression estimation on a bounded interval, optimal rates of decrease for integrated mean square error are known but not the best possible constants. A sharp result on such a constant, i.e., an analog of Fisher's bound for asymptotic variances is obtained for minimax risk over a Sobolev smoothness class. Normality of errors is assumed. The method is based on applying a recent result on minimax filtering in Hilbert space. A variant of spline smoothing is…
Linear estimation is considered in nonparametric regression models of the form $Y_i = f(x_i) + \varepsilon_i, x_i \in (a, b)$, where the zero mean errors are uncorrelated with common variance $\sigma^2$ and the response function $f$ is assumed only to have a bounded square integrable $q$th derivative. The linear estimator which minimizes the maximum mean squared error summed over the observation points is derived, and the exact minimax rate of convergence is obtained. For…
If a discrete-time random walk, consisting of the sum of independent increments from an unknown underlying distribution, is observed at every time instant then it is clear that the underlying distribution can be consistently estimated. If, however, we are restricted to observing only a subset of the times, and if this subset is too sparse, then a central limit effect takes over and only two moments can be consistently estimated. We show that divergence of the sum of the…
When the number of design points goes to infinity, we show that the minimum logit chi-squared estimator of the parameter in a linear logistic regression model for binomial response data is asymptotically normal. We also give conditions under which it is consistent.
Wald's general analytic conditions that imply strong consistency of the approximate maximum likelihood estimators (AMLEs) have been extended by Le Cam, Kiefer and Wolfowitz, Huber, Bahadur, and Perlman. All these conditions use the log likelihood ratio of the type $\log\lbrack f(x, \theta)/f(x, \theta_0)\rbrack$, where $\theta_0$ is the true value of the parameter. However these methods usually fail in the nonparametric case. Thus, in this paper, for each $\theta \neq \theta_0$…
Asymptotic Behavior of M Estimators of p Regression Parameters when p2/n is Large; II. Normal Approximation
In a general linear model, Y=Xβ+R with Y and Rn-dimensional, X a n×p matrix, and βp-dimensional, let β^ be an M estimator of β satisfying 0=∑xiψ(yi−xi′β). Let p→∞ such that (plogn)3/2/n→0. Then maxi∣xi′(β^−β)∣→P0, and it is possible to find a uniform normal approximation for the distribution of β^ under which arbitrary…
Rate of Convergence of One- and Two-Step M-Estimators with Applications to Maximum Likelihood and Pitman Estimators
A one-step version Mn(1) and a two-step version Mn(2) of a general M-estimator Mn are suggested such that Mn−Mn(1)=Op(n−1) and Mn−Mn(2)=Op(n−3/2) for every n1/2-consistent initial estimator and under some regularity conditions. In the special case of maximum likelihood estimation, this among other yields that the second-order efficiency properties of Mn(2) coincide with those of Mn. An application to the Pitman…
This paper examines how a Bayesian decision maker would update his/her probability p for the occurrence of an event A in the light of a number of expert opinions expressed as probabilities q1,⋯,qn of A. It is seen, among other things, that the linear opinion pool, λ0p+∑i=1nλiqi, corresponds to an application of Bayes' Theorem when the decision maker has specified only the mean of the marginal distribution for (q1,⋯,qn)…
We may take observations sequentially from a population with unknown mean θ. After this sampling stage, we are to decide whether θ is greater or less than a known constant ν. The net worth upon stopping is either θ or ν, respectively, minus sampling costs. The objective is to maximize the expected net worth when the probability measure of the observations is a Dirichlet process with parameter α. The stopping problem is shown to be…
Suppose that F0 is a population which is symmetric about zero, so that F(⋅)=F0(⋅−θ) is symmetric about θ. We consider the problem of estimating F0 (shape parameter), both θ and F0, and F based on a random sample from F. First, some asymptotically minimax bounds are obtained. Then, some estimates are constructed which are asymptotically minimax-efficient (the risks of which achieve the minimax bounds uniformly). Furthermore, it…
Minimax Variance M-Estimators in ε-Contamination Models
In the framework of Huber's theory of robust estimation of a location parameter, minimax variance M-estimators are studied for error distributions with densities of the form f(x)=(1−ε)h(x)+εg(x), where g is unknown. A well-known result of Huber (1964) is that when h is strongly unimodal, the least informative density f0=(1−ε)h+εg0 has exponential tails. We study the minimax variance solutions when the known…
A Note on the L1 Consistency of Variable Kernel Estimates
A sample X1,⋯,Xn of i.i.d. Rd-valued random vectors with common density f is used to construct the density estimate fn(x)=(1/n)∑i=1nHni−dK((x−Xi)/Hni), where K is a given density on Rd, and the Hni's are positive functions of n,i and X1,⋯,Xn (but not of x). The Hni's can be thought of as locally adapted smoothing parameters. We give sufficient conditions for the weak convergence to 0 of ∫∣fn−f∣…
A data-driven method of choosing the bandwidth, h, of a kernel density estimator is heuristically motivated by considering modifications of the Kullback-Leibler or pseudo-likelihood cross-validation function. It is seen that this means of choosing h is asymptotically equivalent to taking the h that minimizes some compelling error criteria such as the average squared error and the integrated squared error. Thus, for a given kernel function, the bandwidth can be chosen…
Spline Smoothing in Regression Models and Asymptotic Efficiency in L2
Linear estimation is considered in nonparametric regression models of the form Yi=f(xi)+εi,xi∈(a,b), where the zero mean errors are uncorrelated with common variance σ2 and the response function f is assumed only to have a bounded square integrable qth derivative. The linear estimator which minimizes the maximum mean squared error summed over the observation points is derived, and the exact minimax rate of convergence is obtained. For…
Wald's general analytic conditions that imply strong consistency of the approximate maximum likelihood estimators (AMLEs) have been extended by Le Cam, Kiefer and Wolfowitz, Huber, Bahadur, and Perlman. All these conditions use the log likelihood ratio of the type log[f(x,θ)/f(x,θ0)], where θ0 is the true value of the parameter. However these methods usually fail in the nonparametric case. Thus, in this paper, for each θ=θ0…