Recently, Rissanen proposed a new model selection criterion PLS that selects the model that minimizes the accumulated squares of prediction errors. Usually, the information-based criteria, such as AIC and BIC, select the model that minimizes a loss function which can be expressed as a sum of two terms. One measures the goodness of fit and the other penalizes the complexity of the selected model. In this paper we provide such an interpretation for PLS. Using this…
A general convergence criterion for certain iterative sequences in Hilbert space is presented. For an important subclass of these sequences, estimates of the rate of convergence are given. Under very mild assumptions these results establish an $O(1/ \sqrt n)$ nonsampling convergence rate for projection pursuit regression and neural network training; where $n$ represents the number of ridge functions, neurons or coefficients in a greedy basis expansion.
An approach to bootstrapping kernel spectral density estimates is described which is based on resampling from the periodogram of the original data. We show that it is asymptotically valid under suitable conditions, and we illustrate its performance for a medium-sized time series sample with a small simulation study.
For $i = 1,2,\ldots$, let $X_i$ and $Y_i$ be $\mathbb{R}^d$-valued ($d \geq 1$ integer) and $\mathbb{R}$-valued, respectively, random variables, and let $\{(X_i, Y_i)\}, i \geq 1$, be a strictly stationary and $\alpha$-mixing stochastic process. Set $m(x) = \mathscr{E}(Y_1\mid X_1 = x), x \in \mathbb{R}^d$, and let $\hat{m}_n(x)$ be a certain recursive kernel estimate of $m(x)$. Under suitable regularity conditions and as $n \rightarrow \infty$, it is shown that $\hat{m}_n(x)$…
Krewski and Rao consider inference for a (nonlinear) function of a vector of finite population means $\theta = g(\bar{Y}).$ For a sequence of finite populations with increasing number of strata, they demonstrate that $\hat{\theta} = g(\bar{y})$ is asymptotically normal, where $\bar{y}$ is the usual unbiased stratified estimator of $\bar{Y}.$ Additionally, they demonstrate that $(\hat{\theta} - \theta)/\upsilon^{1/2}(\hat{\theta})$ is asymptotically a standard normal…
In this paper we study the Kaplan-Meier estimator in the case where survival and censoring times are not all i.i.d. We prove several results which are analogous to those shown by van Zuijlen in the complete data case.
In this paper we shall discuss the estimation of the mean of a normal distribution with variance 1. The main question in this work is the existence and computation of a least favorable distribution among all the prior distributions satisfying a given set of constraints. In the following we show that if this distribution is bounded from above on some even moment, then the least favorable distribution exists and it is either normal or discrete. The support of the discrete…
Test statistics which are almost determined by $o(n)$ tail order statistics are shown to provide tests of asymptotic relative efficiency 0 against the usual type of contiguous alternative. The result is applied to several goodness-of-fit tests: the variance weighted Kolmogorov-Smirnov statistic, the Kolmogorov-Smirnov statistic in the stabilized probability plot and the correlation coefficient in a $Q - Q$ plot for a variety of distributions with exponential tails.
T. N. Sriram · I. V. Basawa · R. M. Huggins·刊期 1991-12-01 · 卷 19（4）·The Annals of Statistics已编入正刊
带迁入分枝过程的序贯估计AI 解读
For the critical and subcritical Galton-Watson processes with immigration, it is shown that if the data were collected according to an appropriate stopping rule, the natural sequential estimator of the offspring mean $m$ is asymptotically normally distributed for each fixed $m \in (0, 1\rbrack.$ Furthermore, the sequential estimator is shown to be asymptotically normally distributed uniformly over a class of offspring distributions with $m \in (0, 1\rbrack$ bounded variance…
This paper concerns the distributions used to construct confidence intervals for the regression function in a nonparametric setup. Some rates of convergence for the normal limit, its plug-in approach and the wild bootstrap are obtained conditionally on the explanatory variable $X$ and also unconditionally. The bound found for the wild bootstrap approximation is slightly better (by a factor $n^{-1/45}$) than the bounds given by the plug-in approach or the CLT for the…
This paper considers the problem of constructing approximate confidence intervals for functional parameters in the nonparametric case. The approach based on transformation theory is applied to improve standard confidence intervals. The accelerated bias-corrected percentile interval introduced by Efron relies on the existence of a normalizing transformation with bias and skewness corrections, although calculation does not require explicit knowledge of its functional form. We…
A Bayesian version of Elfving's theorem is given for the $\mathbf{c}$-optimality criterion with emphasis on the inherent geometry. Conditions under which a one-point design is Bayesian $\mathbf{c}$-optimum are described. The class of prior precision matrices $R$ for which the Bayesian $\mathbf{c}$-optimal designs are supported by the points of the classical $\mathbf{c}$-optimal design is characterized. It is proved that the Bayesian $\mathbf{c}$-optimal design, for large $n,$…
Suppose $X_i$ are i.i.d. random variables taking values in $\mathscr{X}, \Theta$ is a parameter space and $y: \mathscr{X} \times \Theta \rightarrow \mathbf{R}$ is a map. Define the averages $S_n(y, \theta) = (1/n)\sum^n_{i = 1}y(X_i, \theta)$ and the truncated expectations $T_m(y, \theta) = \mathbf{E} \max(y(X_1, \theta), - m)$. Under the hypothesis of global dominance [i.e., $\mathbf{E} \sup_\Theta y(X_1, \theta) < \infty$] and some regularity conditions, the main result of…
We derive necessary and sufficient conditions in order that each mixture of a given family of probability densities have no more than $s$ modal intervals, with special attention to ordinary unimodality and strong unimodality of such mixtures.
The RC model has been proposed as a model for ordered contingency tables. It involves parametric scores that are assigned to the rows and columns of the table so that these scores reflect the ordinality of the row and column categories. Efficient estimation of these parameters subject to order constraints remained an open problem mainly due to severe difficulties in computing these estimates and difficulties in deriving an appropriate asymptotic goodness-of-fit test. A…
Confidence intervals for the variance of a normal distribution with unknown mean are constructed which improve upon the usual shortest interval based on the sample variance alone. These intervals have guaranteed coverage probability uniformly greater than a predetermined value $1-\alpha$ and have uniformly shorter length. Using information relating the size of the samples mean to that of the sample variance, we smoothly shift the usual minimum length interval closer to zero,…
One observes $n$ data points, $(\mathbf{t}_i, Y_i),$ with the mean of $Y_i$, conditional on the regression function $f,$ equal to $f(\mathbf{t}_i).$ The prior distribution of the vector $\mathbf{f} = (f(\mathbf{t}_1), \ldots, f(\mathbf{t}_n))^t$ is unknown, but lies in a known class $\Omega.$ An estimator, $\hat{\mathbf{f}},$ of $\mathbf{f}$ is found which minimizes the maximum $E\|\hat{\mathbf{f}} - \mathbf{f}\|^2.$ The maximum is taken over all priors in $\Omega$ and the…
As does Woodroofe, we consider a Bayesian sequential allocation between two treatments that incorporates a covariate. The goal is to maximize the total discounted expected reward from an infinite population of patients. Although our model is more general than Woodroofe's, we are able to duplicate his main result: The myopic rule is asymptotically optimal.
We examine the decision theoretic estimated confidence approach proposed by Kiefer, Robinson and Berger, and focus on results under the frequentist validity constraint previously described by Brown and by Berger. Our main result is that the usual constant coverage probability estimator for the usual confidence set of a linear model is admissible under the frequentist validity constraint. Note that it is inadmissible without the frequentist validity constraint when the…
Ridge regression is a well-known technique to estimate the coefficients of a linear model. The method of regularization is a similar approach commonly used to solve underdetermined linear equations with discrete noisy data. When applying such a technique, the choice of the smoothing (or regularization) parameter $h$ is crucial. Generalized cross-validation (GCV) and Mallows' $C_L$ are two popular methods for estimating a good value for $h,$ from the data. Their asymptotic…
The asymptotically best bandwidth selectors for a kernel density estimator currently require the use of either unappealing higher order kernel pilot estimators or related Fourier transform methods. The point of this paper is to present a methodology which allows the fastest possible rate of convergence with the use of only nonnegative kernel estimators at all stages of the selection process. The essential idea is derived through careful study of factorizations of the pilot…
For nonparametric regression, in the case of dependent observations, cross-validation is known to be severely affected by dependence. This effect is precisely quantified through a limiting distribution for the cross-validated bandwidth. The performance of two methods, the "leave-$(2l + 1)$-out" version of cross-validation and partitioned cross-validation, which adjust for the effect of dependence on bandwidth selection is investigated. The bandwidths produced by these two…
The problem of automatic bandwidth selection for a kernel density estimator is considered. It is well recognized that the bandwidth estimate selected by the least squares cross-validation is subject to large sample variation. This difficulty limits the application of the cross-validation estimate. Based on characteristic functions, an important expression for the cross-validation bandwidth estimate is obtained. The expression clearly points out the source of variation. To…
A general convergence criterion for certain iterative sequences in Hilbert space is presented. For an important subclass of these sequences, estimates of the rate of convergence are given. Under very mild assumptions these results establish an O(1/n) nonsampling convergence rate for projection pursuit regression and neural network training; where n represents the number of ridge functions, neurons or coefficients in a greedy basis expansion.
For i=1,2,…, let Xi and Yi be Rd-valued (d≥1 integer) and R-valued, respectively, random variables, and let {(Xi,Yi)},i≥1, be a strictly stationary and α-mixing stochastic process. Set m(x)=E(Y1∣X1=x),x∈Rd, and let m^n(x) be a certain recursive kernel estimate of m(x). Under suitable regularity conditions and as n→∞, it is shown that m^n(x)…
Krewski and Rao consider inference for a (nonlinear) function of a vector of finite population means θ=g(Yˉ). For a sequence of finite populations with increasing number of strata, they demonstrate that θ^=g(yˉ) is asymptotically normal, where yˉ is the usual unbiased stratified estimator of Yˉ. Additionally, they demonstrate that (θ^−θ)/υ1/2(θ^) is asymptotically a standard normal…
Test statistics which are almost determined by o(n) tail order statistics are shown to provide tests of asymptotic relative efficiency 0 against the usual type of contiguous alternative. The result is applied to several goodness-of-fit tests: the variance weighted Kolmogorov-Smirnov statistic, the Kolmogorov-Smirnov statistic in the stabilized probability plot and the correlation coefficient in a Q−Q plot for a variety of distributions with exponential tails.
For the critical and subcritical Galton-Watson processes with immigration, it is shown that if the data were collected according to an appropriate stopping rule, the natural sequential estimator of the offspring mean m is asymptotically normally distributed for each fixed m∈(0,1]. Furthermore, the sequential estimator is shown to be asymptotically normally distributed uniformly over a class of offspring distributions with m∈(0,1] bounded variance…
This paper concerns the distributions used to construct confidence intervals for the regression function in a nonparametric setup. Some rates of convergence for the normal limit, its plug-in approach and the wild bootstrap are obtained conditionally on the explanatory variable X and also unconditionally. The bound found for the wild bootstrap approximation is slightly better (by a factor n−1/45) than the bounds given by the plug-in approach or the CLT for the…
A Bayesian version of Elfving's theorem is given for the c-optimality criterion with emphasis on the inherent geometry. Conditions under which a one-point design is Bayesian c-optimum are described. The class of prior precision matrices R for which the Bayesian c-optimal designs are supported by the points of the classical c-optimal design is characterized. It is proved that the Bayesian c-optimal design, for large n,…
Suppose Xi are i.i.d. random variables taking values in X,Θ is a parameter space and y:X×Θ→R is a map. Define the averages Sn(y,θ)=(1/n)∑i=1ny(Xi,θ) and the truncated expectations Tm(y,θ)=Emax(y(X1,θ),−m). Under the hypothesis of global dominance [i.e., EsupΘy(X1,θ)<∞] and some regularity conditions, the main result of…
We derive necessary and sufficient conditions in order that each mixture of a given family of probability densities have no more than s modal intervals, with special attention to ordinary unimodality and strong unimodality of such mixtures.
Confidence intervals for the variance of a normal distribution with unknown mean are constructed which improve upon the usual shortest interval based on the sample variance alone. These intervals have guaranteed coverage probability uniformly greater than a predetermined value 1−α and have uniformly shorter length. Using information relating the size of the samples mean to that of the sample variance, we smoothly shift the usual minimum length interval closer to zero,…
One observes n data points, (ti,Yi), with the mean of Yi, conditional on the regression function f, equal to f(ti). The prior distribution of the vector f=(f(t1),…,f(tn))t is unknown, but lies in a known class Ω. An estimator, f^, of f is found which minimizes the maximum E∥f^−f∥2. The maximum is taken over all priors in Ω and the…
Asymptotic Optimality of the Fast Randomized Versions of GCV and CL in Ridge Regression and Regularization
Ridge regression is a well-known technique to estimate the coefficients of a linear model. The method of regularization is a similar approach commonly used to solve underdetermined linear equations with discrete noisy data. When applying such a technique, the choice of the smoothing (or regularization) parameter h is crucial. Generalized cross-validation (GCV) and Mallows' CL are two popular methods for estimating a good value for h, from the data. Their asymptotic…
On Lp-Norms of Multivariate Density Estimators
Central limit theorems are proven for Lp-norms (1≤p<∞) of multivariate density estimators.
For nonparametric regression, in the case of dependent observations, cross-validation is known to be severely affected by dependence. This effect is precisely quantified through a limiting distribution for the cross-validated bandwidth. The performance of two methods, the "leave-(2l+1)-out" version of cross-validation and partitioned cross-validation, which adjust for the effect of dependence on bandwidth selection is investigated. The bandwidths produced by these two…