Sliced inverse regression (SIR), formally introduced by Li, is a very general procedure for performing dimension reduction in nonparametric regression. This paper considers a version of SIR in which the “slices” are determined by nearest neighbors and the response variable takes value possibly in a multidimensional space. It is shown, under general conditions, that the “effective dimension reduction space” can be estimated with rate $n^{-1/2}$ where $n$ is the sample size.
Some sufficient conditions to establish the rate of convergence of certain $M$-estimators in a Gaussian white noise model are presented. They are applied to some concrete problems, including jump point estimation and nonparametric maximum likelihood estimation, for the regression function. The results are shown by means of a maximal inequality for continuous martingales and some techniques developed recently in the context of empirical processes.
For a version of the interval censoring model, case 2, in which the observation intervals are allowed to be arbitrarily small, we consider estimation of functionals that are differentiable along Hellinger differentiable paths. The asymptotic information lower bound for such functionals can be represented as the squared $L_{2}$-norm of the canonical gradient in the observation space. This canonical gradient has an implicit expression as a solution of an integral equation that…
A widely held notion of classical conditional theory is that statistical inference in the presence of ancillary statistics should be independent of the distribution of those ancillary statistcs. In this paper, ancillary paradoxes which contradict this notion are presented for two scenarios involving confidence estimation. These results are related to Brown’s ancillary paradox in point estimation. Moreover, the confidence coefficient, the usual constant coverage probability…
This paper derives an extended version of the Haff or, more appropriately, Stein-Haff identity for an elliptically contoured distribution (ECD) . This identity is then used to show that the minimax estimators of the covariance matrix obtained under normal models remain robust under the ECD model.
In this paper we present a new characterization and perspective on a neutral to the right prior. This characterization is based on a sequence of predictive laws which provides explicitly the posterior parameters and Bayes estimators for such a prior.
This paper is concerned with nonparametric Bayesian inference of the Aalen’s multiplicative counting process model. For a desired nonparametric prior distribution of the cumulative intensity function, a class of Lévy processes is considered, and it is shown that the class of Lévy processes is conjugate for the multiplicative counting process model, and formulas for obtaining a posterior process are derived. Finally, our results are applied to several practically important…
State space models is a very general class of time series models capable of modelling dependent observations in a natural and interpretable way. Inference in such models has been studied by Bickel, Ritov and Rydén, who consider hidden Markov models, which are special kinds of state space models, and prove that the maximum likelihood estimator is asymptotically normal under mild regularity conditions. In this paper we generalize the results of Bickel, Ritov and Rydén to state…
Bandwidth selection for procedures such as kernel density estimation and local regression have been widely studied over the past decade. Substantial “evidence” has been collected to establish superior performance of modern plug-in methods in comparison to methods such as cross validation; this has ranged from detailed analysis of rates of convergence, to simulations, to superior performance on real datasets. In this work we take a detailed look at some of this evidence,…
We study estimation in the class of stationary variable length Markov chains (VLMC) on a finite space. The processes in this class are still Markovian of high order, but with memory of variable length yielding a much bigger and structurally richer class of models than ordinary high-order Markov chains. From an algorithmic view, the VLMC model class has attracted interest in information theory and machine learning, but statistical properties have not yet been explored.…
Andrew Barron · Mark J. Schervish · Larry Wasserman·刊期 1999-04-01 · 卷 27（2）·The Annals of Statistics已编入正刊
非参数问题中后验分布的一致性AI 解读
We give conditions that guarantee that the posterior probability of every Hellinger neighborhood of the true distribution tends to 1 almost surely. The conditions are (1) a requirement that the prior not put high mass near distributions with very rough densities and (2) a requirement that the prior put positive mass in Kullback-Leibler neighborhoods of the true distribution. The results are based on the idea of approximating the set of distributions with a finite-dimensional…
Based on a revised Lancaster-type representation of the additive interactions associated with a probability measure, a new approach for the analysis of high-dimensional contingency tables is proposed. The approach is essentially model-free because the additive interaction tensor is merely a convenient reparameterization of the given table. Single interaction terms are investigated using the bootstrap method whose first-order asymptotic validity is immediate. The global…
Z.Given measurements $(x_i,y_i) = 1,\dots,n$, we discuss methods to assess whether an underlying regression function is smooth (continuous or differentiable) or whether it has discontinuities. The variance of the measurements is assumed to be unknown, and is estimated simultane-ously. By regressing squared differences of the data formed with various span sizes on the span size itself, we obtain an asymptotic linear model with dependent errors. The parameters of this…
We answer two open questions concerning the existence of universal schemes for classification and regression estimation from stationary ergodic processes. It is shown that no measurable procedure can produce weakly consistent regression estimates from every bivariate stationary ergodic process, even if the covariate and response variables are restricted to take values in the unit interval. It is further shown that no measurable procedure can produce weakly consistent…
Suppose that X is a random variable with density $f(x|\theta)$ and that $\pi(\theta|x)$ is a proper posterior corresponding to an improper prior $\nu(\theta)$. The prior is called $\mathscr{P}$-admissible if the generalized Bayes estimator of every bounded function of $\theta$ is almost-$\nu$-admissible under squared error loss. Eaton showed that recurrence of the Markov chain with transition density $R(\eta|\theta) = \int \pi(\eta|x)f(x|\theta) dx$ is a sufficient condition…
The paper is concerned with testing uniformity versus a monotone den-sity. This problem arises in two important contexts, after transformations, testing whether a sample is a simple random sample or a biased sample, and testing whether the intensity function of a nonhomogeneous Poisson process is constant against monotone alternatives. A penalized likelihood ratio test ($P$-test) and a Dip likelihood test ($D$-test) are developed. The $D$-test is analogous to Hartigan and…
For independent $d$-variate random variables $X_1,\dots,X_m$ with common density $f$ and $Y_1,\dots,Y_n$ with common density $g$, let $R_{m,n}$ be the number of edges in the minimal spanning tree with vertices $X_1,\dots,X_m$, $Y_1,\dots,Y_n$ that connect points from different samples. Friedman and Rafsky conjectured that a test of $H_0: f = g$ that rejects $H_0$ for small values of $R_{m,n}$ should have power against general alternatives. We prove that $R_{m,n}$ is…
One of the most difficult problems occurring with stepwise multiple test procedures for a set of two-sided hypotheses is the control of direc-tional errors if rejection of a hypothesis is accomplished with a directional decision. In this paper we generalize a result for so-called step-down procedures derived by Shaffer to a large class of stepwise or closed multiple test procedures. In a unifying way we obtain results for a large class of order statistics procedures…
Motivated by Gaussian tests for a time series, we are led to investigate the asymptotic behavior of the residual empirical processes of stochastic regression models. These models cover the fixed design regression models as well as general AR$(q)$ models. Since the number of the regression coeffi-cients is allowed to grow as the sample size increases, the obtained results are also applicable to nonlinear regression and stationary AR$(\infty)$ models. In this paper, we first…
We consider time series $Y_t = G(X_t)$ where $X_t$ is Gaussian with long memory and $G$ is a polynomial. The series $Y_t$ may or may not have long memory. The spectral density $g_\theta(x)$ of $Y_t$ is parameterized by a vector $\theta$ and we want to estimate its true value $\theta_0$ . We use a least-squares Whittle-type estimator $\hat{\theta}_N$ for $\theta_0$, based on observations $Y_1,\dots,Y_N$. If $Y_t$ is Gaussian, then $\sqrt{N}(\hat{\theta}_N-\theta_0)$ converges…
Z In a class of semiparametric mixture models, the score function (and consequently the effective information) for a finite-dimensional parameter can be made arbitrarily small depending upon the direction taken in the parameter space. This result holds for a broad range of semiparametric mixtures over exponential families and includes examples such as the gamma semiparametric mixture, the normal mean mixture, the Weibull semiparametric mixture and the negative binomial…
This paper presents convergence conditions for a Markov chain constructed using Gibbs sampling, when the equilibrium distribution is the conditional sampling distribution of sufficient statistics from a generalized linear model. For cases when this unidimensional sampling is done approximately rather than exactly, the difference between the target equilibrium distribution and the resulting equilibrium distribution is expressed in terms of the difference between the true and…
We characterize those coherent design criteria which depend only on the dispersion matrix (assumed proper and nonsingular) of the “state of nature,” which may be a parameter-vector or a set of future observables, and describe the associated decision problems. Connections are established with the classical approach to optimal design theory for the normal linear model, based on concave functions of the information matrix. Implications of the theory for more general models are…
The quasi-likelihood function proposed by Wedderburn broadened the scope of generalized linear models by specifying the variance function in-stead of the entire distribution. However, complete specification of variance functions in the quasi-likelihood approach may not be realistic. We define a nonparametric quasi-likelihood by replacing the specified variance function in the conventional quasi-likelihood with a nonparametric variance function estimate. This nonparametric…
Sliced inverse regression (SIR), formally introduced by Li, is a very general procedure for performing dimension reduction in nonparametric regression. This paper considers a version of SIR in which the “slices” are determined by nearest neighbors and the response variable takes value possibly in a multidimensional space. It is shown, under general conditions, that the “effective dimension reduction space” can be estimated with rate n−1/2 where n is the sample size.
A maximal inequality for continuous martingales and M-estimation in a Gaussian white noise model
Some sufficient conditions to establish the rate of convergence of certain M-estimators in a Gaussian white noise model are presented. They are applied to some concrete problems, including jump point estimation and nonparametric maximum likelihood estimation, for the regression function. The results are shown by means of a maximal inequality for continuous martingales and some techniques developed recently in the context of empirical processes.
Asymptotically optimal estimation of smooth functionals for interval censoring, case 2
For a version of the interval censoring model, case 2, in which the observation intervals are allowed to be arbitrarily small, we consider estimation of functionals that are differentiable along Hellinger differentiable paths. The asymptotic information lower bound for such functionals can be represented as the squared L2-norm of the canonical gradient in the observation space. This canonical gradient has an implicit expression as a solution of an integral equation that…
Z.Given measurements (xi,yi)=1,…,n, we discuss methods to assess whether an underlying regression function is smooth (continuous or differentiable) or whether it has discontinuities. The variance of the measurements is assumed to be unknown, and is estimated simultane-ously. By regressing squared differences of the data formed with various span sizes on the span size itself, we obtain an asymptotic linear model with dependent errors. The parameters of this…
Eaton's Markov chain, its conjugate partner and P-admissibility
Suppose that X is a random variable with density f(x∣θ) and that π(θ∣x) is a proper posterior corresponding to an improper prior ν(θ). The prior is called P-admissible if the generalized Bayes estimator of every bounded function of θ is almost-ν-admissible under squared error loss. Eaton showed that recurrence of the Markov chain with transition density R(η∣θ)=∫π(η∣x)f(x∣θ)dx is a sufficient condition…
The paper is concerned with testing uniformity versus a monotone den-sity. This problem arises in two important contexts, after transformations, testing whether a sample is a simple random sample or a biased sample, and testing whether the intensity function of a nonhomogeneous Poisson process is constant against monotone alternatives. A penalized likelihood ratio test (P-test) and a Dip likelihood test (D-test) are developed. The D-test is analogous to Hartigan and…
For independent d-variate random variables X1,…,Xm with common density f and Y1,…,Yn with common density g, let Rm,n be the number of edges in the minimal spanning tree with vertices X1,…,Xm, Y1,…,Yn that connect points from different samples. Friedman and Rafsky conjectured that a test of H0:f=g that rejects H0 for small values of Rm,n should have power against general alternatives. We prove that Rm,n is…
Motivated by Gaussian tests for a time series, we are led to investigate the asymptotic behavior of the residual empirical processes of stochastic regression models. These models cover the fixed design regression models as well as general AR(q) models. Since the number of the regression coeffi-cients is allowed to grow as the sample size increases, the obtained results are also applicable to nonlinear regression and stationary AR(∞) models. In this paper, we first…
We consider time series Yt=G(Xt) where Xt is Gaussian with long memory and G is a polynomial. The series Yt may or may not have long memory. The spectral density gθ(x) of Yt is parameterized by a vector θ and we want to estimate its true value θ0 . We use a least-squares Whittle-type estimator θ^N for θ0, based on observations Y1,…,YN. If Yt is Gaussian, then N(θ^N−θ0) converges…