We consider the problem of sharp-optimal estimation of a response function $f(x)$ in a random design nonparametric regression under a general model where a pair of observations $(Y, X)$ has a joint density $p(y, x) = p(y|f(x)) \pi(x)$. We wish to estimate the response function with optimal minimax mean integrated squared error convergence as the sample size tends to $\infty$. Traditional regularity assumptions on the conditional density $p(y| \theta)$ assumed for parameter $\theta$…
Minimax mean-squared error estimates of quadratic functionals of smooth functions have been constructed for a variety of smoothness classes. In contrast to many nonparametric function estimation problems there are both regular and irregular cases. In the regular cases the minimax mean-squared error converges at a rate proportional to the inverse of the sample size, whereas in the irregular case much slower rates are the rule. We investigate the problem of adaptive estimation…
We establish general sufficient conditions for the $L_2$-consistency of multivariate histogram regression estimates based on data-dependent partitions. These same conditions insure the consistency of partitioning regression estimates based on local polynomial fits, and, with an additional regularity assumption, the consistency of histogram estimates for conditional medians. Our conditions require shrinking cells, subexponential growth of a combinatorial complexity measure…
A large-deviations criterion is proposed for optimality of nonparametric regression estimators. The criterion is one of minimaxity of the large-deviations probabilities. We study the case where the underlying class of regression functions is either Lipschitz or Hölder, and when the loss function involves estimation at a point or in supremum norm. Exact minimax asymptotics are found in the Gaussian case.
Jen-Fue Maa · Dennis K. Pearl · Robert Bartoszyński·刊期 1996-06-01 · 卷 24（3）·The Annals of Statistics已编入正刊
将多维两样本数据化为一维点间比较AI 解读
The most popular technique for reducing the dimensionality in comparing two multidimensional samples of $\mathbf{X} \sim F$ and $\mathbf{Y}\sim G$ is to analyze distributions of interpoint comparisons based on a univariate function h (e.g. the interpoint distances). We provide a theoretical foundation for this technique, by showing that having both i) the equality of the distributions of within sample comparisons $(h(\mathbf{X}_1, \mathbf{X}_2) =_L h(\mathbf{Y}_1, \mathbf{Y}_2))$…
To explore nonlinear structures hidden in high-dimensional data and to estimate the effective dimension reduction directions in multivariate nonparametric regression, Li and Duan proposed the sliced inverse regression (SIR) method which is simple to use. In this paper, the asymptotic properties of the kernel estimate of sliced inverse regression are investigated. It turns out that regardless of the kernel function, the asymptotic distribution remains the same for a wide…
This paper proves the local asymptotic normality of a stationary and ergodic first order random coefficient autoregressive model in a semiparametric setting. This result is used to show that Stein's necessary condition for adaptive estimation of the mean of the random coefficient is satisfied if the distributions of the innovations and the errors in the random coefficients are symmetric around zero. Under these symmetry assumptions, a locally asymptotically minimax adaptive…
Let $X_n = \Sigma_{i=1}^{\infty} a_i \varepsilon_{n-i}$, where the $\varepsilon_i$ are iid with mean 0 finite fourth moment and the $a_i$ are regularly varying with index $-\beta$ where $\beta \epsilon (1/2, 1)$ so that ${X_n}$ has long-range dependence. This covers an important class of the fractional ARIMA process. For $r \geq 0$, let $Y_{N, r} = \sum_{n=1}^N \sum_{1\leq j_1 < \dots < j_r} \Pi_{s=1}^r a_{j_s}, Y_{N, 0} = N, \sigma_{N, r}^2 = \Var(Y_{N, r})$ and $F^{(r)} =$…
A concept of asymptotically efficient estimation is presented when a misspecified parametric time series model is fitted to a stationary process. Efficiency of several minimum distance estimates is proved and the behavior of the Gaussian maximum likelihood estimate is studied. Furthermore, the behavior of estimates that minimize the h-step prediction error is discussed briefly. The paper answers to some extent the question what happens when a misspecified model is fitted to…
In hierarchical Bayesian modeling of normal means, it is common to complete the prior specification by choosing a constant prior density for unmodeled hyperparameters (e.g., variances and highest-level means). This common practice often results in an inadequate overall prior, inadequate in the sense that estimators resulting from its use can be inadmissible under quadratic loss. In this paper, hierarchical priors for normal means are categorized in terms of admissibility and…
Let X be a random vector uniformly distributed on the unit cube and $f: [0, 1]^3 \to \mathsf{R}$ be a measurable function. An objective of many computer experiments is to estimate $\mu = E(f \circ X)$ by computing f at a set of points in $[0, 1]^3$. There is a design issue in choosing these points. Recently Owen and Tang independently suggested using randomized orthogonal arrays in the choice of such a set. This paper investigates the convergence rate to normality of the…
Lanh Tran · George Roussas · Sidney Yakowitz · B. Truong Van·刊期 1996-06-01 · 卷 24（3）·The Annals of Statistics已编入正刊
线性时间序列下的固定设计回归AI 解读
This investigation is concerned with recovering a regression function $g(x_i)$ on the basis of noisy observations taken at uniformly spaced design points $x_i$. It is presumed that the corresponding observations are corrupted by additive dependent noise, and that the noise is, in fact, induced by a general linear process in which the summand law can be discrete, as well as continuously distributed. Discreteness induces a complication because such noise is not known to be…
We consider the empirical Bayes estimation of a distribution using binary data via the Dirichlet process. Let $\mathscr{D}(\alpha)$ denote a Dirichlet process with $\alpha$ being a finite measure on Instead of having direct samples from an unknown random distribution F from $\mathscr{D}(\alpha)$, we assume that only indirect binomial data are observable. This paper presents a new interpretation of Lo's formula, and thereby relates the predictive density of the observations…
In this paper results and counterexamples are given to study the relationship between some conditions which appear in the literature on sufficiency, invariance and conditional independence.
In this paper we show that the well-known asymptotic efficiency bounds for full mixture models remain valid if individual sequences of nuisance parameters are considered. This is made precise both for some classes of random (i.i.d.) and nonrandom nuisance parameters. For the random case it is shown that superefficiency of the kind given by an example of Pfanzagl can happen only with low probability. The nonrandom case deals with permutation-invariant estimators under…
We consider maximum likelihood estimation in several examples of semiparametric mixture models, including the exponential frailty model and the errors-in-variables model. The observations consist of a sample of size n from the mixture density $\int p_{\theta}(x|z) d \eta(z)$. The mixing distribution is completely unknown. We show that the first component $\hat{\theta}_n$ of the joint maximum likelihood estimator , $(\hat{\theta}_n \hat{\eta}_n)$ is asymptotically normal and…
Tests of hypotheses about finite-dimensional parameters in a semiparametric model are studied from Pitman's moving alternative (or local) approach using Le Cam's local asymptotic normality concept. For the case of a real parameter being tested, asymptotically uniformly most powerful (AUMP) tests are characterized for one-sided hypotheses, and AUMP unbiased tests for two-sided ones. An asymptotic invariance principle is introduced for multidimensional hypotheses, and AUMP…
A two-way multivariate normal model is proposed and attention is focused on estimation of the mean values when the common variance of the observations is unknown. A class of empirical Bayes estimators is proposed and mean-squared errors are given. A lower bound on the mean-squared error is found and related to risk asymptotics. A James-Stein-type estimator is derived and compared with its competitor--a modal estimator that is obtained from a hierarchical prior for the…
We give a geometric proof that the estimates of a regression model derived by using partial least squares shrink the ordinary least squares estimates. The proof is based on a sequential construction algorithm of partial least squares. A discussion of the nature of shrinkage is included.
For a broad class of error distributions that includes the spherically symmetric ones, we give a short proof that the usual estimator of the mean in a d-dimensional shift model is inadmissible under quadratic loss when $d \geq 3$. Our proof involves representing the error distribution as that of a stopped Brownian motion and using elementary stochastic analysis to obtain a generalization of an integration by parts lemma due to Stein in the Gaussian case.
We discuss maximum likelihood methods for fitting a broad class of multivariate categorical response data models. In particular, we derive the large-sample distributions for maximum likelihood estimators of parameters of product-multinomial generalized log-linear models. The large-sample behavior of other relevant likelihood-based statistics such as goodness-of-fit statistics and adjusted residuals is also described. The asymptotic results are derived within the framework of…
Bickel and Ritov suggested an optimal estimator for the integral of the square of the kth derivative of a density when the unknown density belongs to a Lipschitz class of a given order $\beta$. In this context optimality means that the estimate is asymptotically efficient, that is, it has the best constant and rate of risk convergence, whenever $\beta > 2k + 1/4$, and it is rate optimal otherwise. The suggested optimal estimator crucially depends on the value of $\beta$ which…
We consider the problem of estimating a functional of a density of the type $\int \phi (f, \cdot)$. Starting from efficient estimators of linear and quadratic functionals of f and using a Taylor expansion of $\phi$, we build estimators that achieve the $n^{-1/2}$ rate whenever f is smooth enough. Moreover, we show that these estimators are efficient. Concerning the estimation of quadratic functionals (more precisely, of integrated squared density) Bickel and Ritov have…
A model of interval censorship of a failure time T is considered when there is only one inspection time Y. The observable data are n independent copies of the pair $(Y, \delta)$, where $\delta = [T \leq Y]$. We construct a class of self-consistent estimators of the survival function of T defined implicitly through two equations and show their strong consistency under certain conditions. The properties of the onparametric maximum likelihood estimator are also investigated.
We consider the problem of sharp-optimal estimation of a response function f(x) in a random design nonparametric regression under a general model where a pair of observations (Y,X) has a joint density p(y,x)=p(y∣f(x))π(x). We wish to estimate the response function with optimal minimax mean integrated squared error convergence as the sample size tends to ∞. Traditional regularity assumptions on the conditional density p(y∣θ) assumed for parameter θ…
We establish general sufficient conditions for the L2-consistency of multivariate histogram regression estimates based on data-dependent partitions. These same conditions insure the consistency of partitioning regression estimates based on local polynomial fits, and, with an additional regularity assumption, the consistency of histogram estimates for conditional medians. Our conditions require shrinking cells, subexponential growth of a combinatorial complexity measure…
The most popular technique for reducing the dimensionality in comparing two multidimensional samples of X∼F and Y∼G is to analyze distributions of interpoint comparisons based on a univariate function h (e.g. the interpoint distances). We provide a theoretical foundation for this technique, by showing that having both i) the equality of the distributions of within sample comparisons (h(X1,X2)=Lh(Y1,Y2))…
Let Xn=Σi=1∞aiεn−i, where the εi are iid with mean 0 finite fourth moment and the ai are regularly varying with index −β where βϵ(1/2,1) so that Xn has long-range dependence. This covers an important class of the fractional ARIMA process. For r≥0, let $Y_{N, r} = \sum_{n=1}^N \sum_{1\leq j_1 < \dots < j_r} \Pi_{s=1}^r a_{j_s}, Y_{N, 0} = N, \sigma_{N, r}^2 = \Var(Y_{N, r})$ and F(r)=…
Let X be a random vector uniformly distributed on the unit cube and f:[0,1]3→R be a measurable function. An objective of many computer experiments is to estimate μ=E(f∘X) by computing f at a set of points in [0,1]3. There is a design issue in choosing these points. Recently Owen and Tang independently suggested using randomized orthogonal arrays in the choice of such a set. This paper investigates the convergence rate to normality of the…
This investigation is concerned with recovering a regression function g(xi) on the basis of noisy observations taken at uniformly spaced design points xi. It is presumed that the corresponding observations are corrupted by additive dependent noise, and that the noise is, in fact, induced by a general linear process in which the summand law can be discrete, as well as continuously distributed. Discreteness induces a complication because such noise is not known to be…
We consider the empirical Bayes estimation of a distribution using binary data via the Dirichlet process. Let D(α) denote a Dirichlet process with α being a finite measure on Instead of having direct samples from an unknown random distribution F from D(α), we assume that only indirect binomial data are observable. This paper presents a new interpretation of Lo's formula, and thereby relates the predictive density of the observations…
We consider maximum likelihood estimation in several examples of semiparametric mixture models, including the exponential frailty model and the errors-in-variables model. The observations consist of a sample of size n from the mixture density ∫pθ(x∣z)dη(z). The mixing distribution is completely unknown. We show that the first component θ^n of the joint maximum likelihood estimator , (θ^nη^n) is asymptotically normal and…
For a broad class of error distributions that includes the spherically symmetric ones, we give a short proof that the usual estimator of the mean in a d-dimensional shift model is inadmissible under quadratic loss when d≥3. Our proof involves representing the error distribution as that of a stopped Brownian motion and using elementary stochastic analysis to obtain a generalization of an integration by parts lemma due to Stein in the Gaussian case.
Bickel and Ritov suggested an optimal estimator for the integral of the square of the kth derivative of a density when the unknown density belongs to a Lipschitz class of a given order β. In this context optimality means that the estimate is asymptotically efficient, that is, it has the best constant and rate of risk convergence, whenever β>2k+1/4, and it is rate optimal otherwise. The suggested optimal estimator crucially depends on the value of β which…
We consider the problem of estimating a functional of a density of the type ∫ϕ(f,⋅). Starting from efficient estimators of linear and quadratic functionals of f and using a Taylor expansion of ϕ, we build estimators that achieve the n−1/2 rate whenever f is smooth enough. Moreover, we show that these estimators are efficient. Concerning the estimation of quadratic functionals (more precisely, of integrated squared density) Bickel and Ritov have…
A model of interval censorship of a failure time T is considered when there is only one inspection time Y. The observable data are n independent copies of the pair (Y,δ), where δ=[T≤Y]. We construct a class of self-consistent estimators of the survival function of T defined implicitly through two equations and show their strong consistency under certain conditions. The properties of the onparametric maximum likelihood estimator are also investigated.