We construct a tractable mathematical model for kernel-based projection pursuit regression approximation. The model permits computation of explicit formulae for bias and variance of estimators. It is shown that the bias of an orientation estimate dominates error about the mean--indeed, the latter is asymptotically negligible in comparison with bias. However, bias and error about the mean are of the same order in the case of projection pursuit curve estimates. Implications of…
Given data $y_i = (Kg)(u_i) + \varepsilon_i$ where the $\varepsilon$'s are random errors, the $u$'s are known, $g$ is an unknown function in a reproducing kernel space with kernel $r$ and $K$ is a known integral operator, it is shown how to calculate convergence rates for the regularized solution of the equation as the evaluation points $\{u_i\}$ become dense in the interval of interest. These rates are shown to depend on the eigenvalue asymptotics of $KRK^\ast$, where $R$ is…
The aim of this article is to investigate the asymptotic behaviour of estimators of the optimal value and optimal solutions of a stochastic program. These estimators are closely related to the $M$-estimators introduced by Huber (1964). The parameter set of feasible solutions is supposed to be defined by a number of equality and inequality constraints. It will be shown that in the presence of inequality constraints the estimators are not asymptotically normal in general.…
It is shown that the Bayesian bootstrap approximation to the posterior distribution of the unknown mean (with respect to a Dirichlet process prior) is more accurate than both the standard normal approximation and the bootstrap approximation. It is also shown that the Bayesian bootstrap approximation to the sampling distribution of the sample average is not as accurate as the bootstrap approximation.
The desire to make nonparametric regression robust leads to the problem of conditional median function estimation. Under appropriate regularity conditions, a sequence of local median estimators can be chosen to achieve the optimal rate of convergence $n^{-1/(2+d)}$ both pointwise and in the $L^q (1 \leq q < \infty)$ norm restricted to a compact. It can also be chosen to achieve the optimal rate of convergence $(n^{-1} \log n)^{1/(2+d)}$ in the $L^\infty$ norm restricted to a…
Andreas Buja · Trevor Hastie · Robert Tibshirani·刊期 1989-06-01 · 卷 17(2)·The Annals of Statistics已编入正刊
线性平滑器与加性模型AI 解读
We study linear smoothers and their use in building nonparametric regression models. In the first part of this paper we examine certain aspects of linear smoothers for scatterplots; examples of these are the running-mean and running-line, kernel and cubic spline smoothers. The eigenvalue and singular value decompositions of the corresponding smoother matrix are used to describe qualitatively a smoother, and several other topics such as the number of degrees of freedom of a…
Let $X$ be a random vector with at least one marginal having a lattice distribution. For a wide class of statistics which can be written as a function of means of independent copies of $X$, it is established in this article that the one-term Edgeworth expansion is typically the same as the usual one-term expansion in the pure nonlattice case.
Let $\phi$ be a parameter for which there is no unbiased estimator. This note shows that for an arbitrary sequence of estimators $T^{(k)}$, if the biases of $T^{(k)}$ tend to 0 then their variances must tend to $\infty$.
This paper concerns a partial ordering of positive dependence of a class of random variables which have a common marginal distribution and are not necessarily exchangeable. The main theorem is obtained by applying a moment inequality via majorization. Inequalities for exchangeable random variables, for random variables whose marginal densities possess the semigroup property and for the multivariate normal distribution are then obtained as special cases.
The characteristic equation of a multiple autoregressive time series involves the eigenvalues of a matrix equation which determine if the series is stationary. Suppose one eigenvalue is 1 and the rest are less than 1 in magnitude. We show that ordinary least squares may be used to estimate the matrices involved and that the largest estimated eigenvalue has distributional properties that allow us to test this unit root hypothesis using critical values tabulated by Dickey…
This paper aims at improvement in regularity conditions on spectral densities for the limit theorems of the quasi maximum-likelihood estimator and the quasi likelihood-ratio statistic. The approach parallels the Daniels-Huber-Pollard proofs of central limit theorems under nonstandard conditions for i.i.d. situations. The results of the paper enable one to dispense with excessive regularity conditions on the spectral density.
A stochastic expansion for $M$-estimates in linear models with many parameters is derived under the weak condition $\kappa n^{1/3}(\log n)^{2/3} \rightarrow 0$, where $n$ is the sample size and $\kappa$ the maximal diagonal element of the hat matrix. The expansion is used to study the asymptotic distribution of linear contrasts and the consistency of the bootstrap. In particular, it turns out that bootstrap works in cases where the usual asymptotic approach fails.
Asymptotically efficient (adaptive) estimators for the slope parameters of the linear regression model are constructed based upon the "regression quantile" statistics suggested by Koenker and Bassett. The estimators are natural analogues of the adaptive $L$-estimators of location of Sacks, but employ kernel-density type estimators of the optimal $L$-estimator weight function.
We relate the asymptotic behavior of $M$-estimators of the regression parameter in a linear model in which the dimension of the regression parameter may increase with the sample size to the stochastic equicontinuity of an associated $M$-process. The approach synthesises a number of results for the dimensionally fixed regression model and then extends these results in a direct unified way. The resulting theorems require only mild conditions on the $\psi$-function and the…
In the context of Huber's theory of robust estimation of a location parameter, the literature on minimax properties of $M$-, $R$- and $L$-estimators is surveyed. New results are obtained for the model in which the unknown error distribution is assumed to lie in a Levy neighbourhood of a symmetric distribution $G: \mathscr{P}_{\varepsilon,\delta}(G) = \{F\mid G(x - \delta) - \varepsilon \leq F(x) \leq G(x + \delta) + \varepsilon \text{for all} x\}$. Under reasonably general…
A class of invariant hypothesis tests is considered for the purpose of testing a variance ratio arising in mixed models. Members of the class are most powerful for specific alternatives and limiting members of the class are Wald's test and the locally most powerful test. It is demonstrated that the locally most powerful test has the highest and Wald's test has the lowest asymptotic power when an asymptotically unbalanced sequence of ANOVA designs is considered under Pitman…
Suppose $u_1, \cdots, u_n, v_1, \cdots, v_n$ are random points on the sphere such that for unknown points $\xi_1, \cdots, \xi_n$ and unknown rotation $A_0$, the distribution of $u_i$ depends only on $u^t_i\xi_i$ and that of $v_i$ on $v^t_iA_0\xi_i$. This paper provides asymptotic tests and confidence regions for $A_0$ and for its axis of rotation. Results are given in arbitrary dimension.
Multivariate errors-in-variables regression models with normal errors are considered and residuals, similar to those calculated from ordinary least squares regressions, are defined for these models. It is shown that under the assumption of a $n^{1/2}$-consistent estimator of the vector of regression coefficients, certain empirical processes based upon the residuals converge to the same Gaussian process as that of an infinite sequence of normal random variables standardized…
Roger H. Farrell · Witold Klonecki · Stefan Zontek·刊期 1989-03-01 · 卷 17(1)·The Annals of Statistics已编入正刊
伽马尺度参数向量的全部可容许线性估计量及随机效应应用AI 解读
The paper is devoted to the problem of simultaneous estimation of scale and natural parameters of the multiparameter gamma distribution under a quadratic loss. The vector of the scale parameters is assumed to range over a certain subset of the Cartesian product $\mathscr{R}^n_+$ of $n$ positive half lines. We identify the class of all linear admissible estimators for the scale parameters and show that all linear estimators of the natural parameters are inadmissible. Since…
Assume the standard linear model $X_{n \times 1} = A_{n \times p} \theta_{p \times 1} + \varepsilon_{n \times 1},$ where $\varepsilon$ has an $n$-variate normal distribution with zero mean vector and identity covariance matrix. The least squares estimator for the coefficient $\theta$ is $\hat{\theta} \equiv (A'A)^{-1}A'X$. It is well known that $\hat{\theta}$ is dominated by James-Stein type estimators under the sum of squared error loss $|\theta - \hat{\theta}|^2$ when $p \geq 3$…
Consider the one-way fixed and balanced analysis of variance model under the assumptions of independence and normality. The problem is to test for homogeneity of variances. A necessary and sufficient condition for admissibility of a test among the class of scale invariant tests is given. Hartley's test and Cochran's test are not in the class and therefore are inadmissible. Various scale invariant tests are examined for parameter consistency. Parameter consistency in this…
The bootstrap estimate of distribution functions of studentized statistics is shown to be more accurate than even the two-term empirical Edgeworth expansion, thus strengthening the claim of superiority of the bootstrap over the normal approximation method. The two methods are compared not only with respect to bounded bowl-shaped loss functions but also with respect to squared error loss and, more generally, in $L^p$-norms.
In this paper, the asymptotic behavior of some nonparametric tests is studied in situations where both bootstrap tests and randomization tests are applicable. Under fairly general conditions, the tests are asymptotically equivalent in the sense that the resulting critical values and power functions are appropriately close. This implies, among other things, that the difference in the critical functions of the tests, evaluated at the observed data, tends to 0 in probability.…
Stochastic procedures are randomized statistical procedures which are functions of the observed sample and of one or more artificially constructed auxiliary samples. As the size of the auxiliary samples increases, a stochastic procedure becomes nearly nonrandomized. The stochastic test of this paper arises as a numerically feasible approximation to a natural minimum distance goodness-of-fit test for multivariate parametric models. The distance being minimized here is the…
Given data yi=(Kg)(ui)+εi where the ε's are random errors, the u's are known, g is an unknown function in a reproducing kernel space with kernel r and K is a known integral operator, it is shown how to calculate convergence rates for the regularized solution of the equation as the evaluation points {ui} become dense in the interval of interest. These rates are shown to depend on the eigenvalue asymptotics of KRK∗, where R is…
The aim of this article is to investigate the asymptotic behaviour of estimators of the optimal value and optimal solutions of a stochastic program. These estimators are closely related to the M-estimators introduced by Huber (1964). The parameter set of feasible solutions is supposed to be defined by a number of equality and inequality constraints. It will be shown that in the presence of inequality constraints the estimators are not asymptotically normal in general.…
The desire to make nonparametric regression robust leads to the problem of conditional median function estimation. Under appropriate regularity conditions, a sequence of local median estimators can be chosen to achieve the optimal rate of convergence n−1/(2+d) both pointwise and in the Lq(1≤q<∞) norm restricted to a compact. It can also be chosen to achieve the optimal rate of convergence (n−1logn)1/(2+d) in the L∞ norm restricted to a…
Let X be a random vector with at least one marginal having a lattice distribution. For a wide class of statistics which can be written as a function of means of independent copies of X, it is established in this article that the one-term Edgeworth expansion is typically the same as the usual one-term expansion in the pure nonlattice case.
Let ϕ be a parameter for which there is no unbiased estimator. This note shows that for an arbitrary sequence of estimators T(k), if the biases of T(k) tend to 0 then their variances must tend to ∞.
A stochastic expansion for M-estimates in linear models with many parameters is derived under the weak condition κn1/3(logn)2/3→0, where n is the sample size and κ the maximal diagonal element of the hat matrix. The expansion is used to study the asymptotic distribution of linear contrasts and the consistency of the bootstrap. In particular, it turns out that bootstrap works in cases where the usual asymptotic approach fails.
Asymptotically efficient (adaptive) estimators for the slope parameters of the linear regression model are constructed based upon the "regression quantile" statistics suggested by Koenker and Bassett. The estimators are natural analogues of the adaptive L-estimators of location of Sacks, but employ kernel-density type estimators of the optimal L-estimator weight function.
On M-Processes and M-Estimation
We relate the asymptotic behavior of M-estimators of the regression parameter in a linear model in which the dimension of the regression parameter may increase with the sample size to the stochastic equicontinuity of an associated M-process. The approach synthesises a number of results for the dimensionally fixed regression model and then extends these results in a direct unified way. The resulting theorems require only mild conditions on the ψ-function and the…
Minimax Properties of M−,R− and L-Estimators of Location in Levy Neighbourhoods
In the context of Huber's theory of robust estimation of a location parameter, the literature on minimax properties of M-, R- and L-estimators is surveyed. New results are obtained for the model in which the unknown error distribution is assumed to lie in a Levy neighbourhood of a symmetric distribution G:Pε,δ(G)={F∣G(x−δ)−ε≤F(x)≤G(x+δ)+εfor allx}. Under reasonably general…
Suppose u1,⋯,un,v1,⋯,vn are random points on the sphere such that for unknown points ξ1,⋯,ξn and unknown rotation A0, the distribution of ui depends only on uitξi and that of vi on vitA0ξi. This paper provides asymptotic tests and confidence regions for A0 and for its axis of rotation. Results are given in arbitrary dimension.
Multivariate errors-in-variables regression models with normal errors are considered and residuals, similar to those calculated from ordinary least squares regressions, are defined for these models. It is shown that under the assumption of a n1/2-consistent estimator of the vector of regression coefficients, certain empirical processes based upon the residuals converge to the same Gaussian process as that of an infinite sequence of normal random variables standardized…
The paper is devoted to the problem of simultaneous estimation of scale and natural parameters of the multiparameter gamma distribution under a quadratic loss. The vector of the scale parameters is assumed to range over a certain subset of the Cartesian product R+n of n positive half lines. We identify the class of all linear admissible estimators for the scale parameters and show that all linear estimators of the natural parameters are inadmissible. Since…
Universal Domination and Stochastic Domination: U-Admissibility and U- Inadmissibility of the Least Squares Estimator
Assume the standard linear model Xn×1=An×pθp×1+εn×1, where ε has an n-variate normal distribution with zero mean vector and identity covariance matrix. The least squares estimator for the coefficient θ is θ^≡(A′A)−1A′X. It is well known that θ^ is dominated by James-Stein type estimators under the sum of squared error loss ∣θ−θ^∣2 when p≥3…
Second Order and Lp-Comparisons between the Bootstrap and Empirical Edgeworth Expansion Methodologies
The bootstrap estimate of distribution functions of studentized statistics is shown to be more accurate than even the two-term empirical Edgeworth expansion, thus strengthening the claim of superiority of the bootstrap over the normal approximation method. The two methods are compared not only with respect to bounded bowl-shaped loss functions but also with respect to squared error loss and, more generally, in Lp-norms.