The Robbins-Monro process $X_{n+1} = X_n - c_n Y_n$ is a standard stochastic approximation method for estimating the root $\theta$ of an unknown regression function. There is a vast literature on the convergence properties of $X_n$ to $\theta$. In practice, one is also interested in the conditional distribution of the system under the sequential control when the control is set at $\theta$ or near $\theta$. This problem appears to have received no attention in the literature.…
The problem of combining high efficiency with high breakdown properties for regression estimators has piqued the interest of statisticians for some time. One proposal specifically suggested by Rousseeuw and Leroy is to use the least median of squares estimator, omit observations whose residuals are larger than some constant cut-off value and apply least squares to the remaining observations. Although this proposal does retain high breakdown point, it actually converges no…
Rafail Z. Khas'minskii · Betty V. Lazareva·刊期 1992-12-01 · 卷 20（4）·The Annals of Statistics已编入正刊
高斯白噪声观测下跳跃马尔可夫过程的一种滤波方法AI 解读
The importance of optimal filtration problem for Markov chain with two states observed in Gaussian white noise (GWN) for a lot of concrete technical problems is well known. The equation for a posterior probability $\pi(t)$ of one of the states was obtained many years ago. The aim of this paper is to study a simple filtration method. It is shown that this simplified filtration is asymptotically efficient in some sense if the diffusion constant of the GWN goes to 0. Some…
It is well known that anomalies are sometimes observed when using the likelihood ratio test (LRT) for testing restricted hypotheses in a normal model. This paper considers a general framework for these anomalies to occur. We provide a condition, that relates the null and the alternative hypotheses, under which the dominance of the LRT is obtained. Conditions are also given which guarantee the equivalence between the LRT and a simpler test. The situations of known and unknown…
Two new statistics for testing goodness-of-fit are derived from the viewpoint of nonparametric density estimation. These statistics are closely related to the Neyman smooth and Cramer-von Mises statistics but are shown to have superior properties both through asymptotic and small sample analyses. Comparison of the proposed tests with the Cramer-von Mises statistic requires the development of a novel technique for comparing tests that are capable of detecting local…
For the data based choice of the bandwidth of a kernel density estimator, several methods have recently been proposed which have a very fast asymptotic rate of convergence to the optimal bandwidth. In particular the relative rate of convergence is the square root of the sample size, which is known to be the best possible. The point of this paper is to show how semiparametric arguments can be employed to calculate the best possible constant coefficient, that is, an analog of…
We consider the Akaike-Parzen-Rosenblatt density estimate $f_{nh}$ based upon any superkernel $L$ (i.e., an absolutely integrable function with $\int L = 1$, whose characteristic function is 1 on $\lbrack -1, 1\rbrack)$, and compare it with a kernel estimate $g_{nh}$ based upon an arbitrary kernel $K$. We show that for a given subclass of analytic densities, $\inf_L \sup_K \lim \sup_{n\rightarrow \infty} \frac{\inf_h \mathbb{E} \int |f_{nh} - f|}{\inf_h \mathbb{E} \int |g_{nh} - f |} = 1,$…
Random coefficient regression models are important in representing linear models with heteroscedastic errors and in unifying the study of classical fixed effects and random effects linear models. For prediction intervals and for bootstrapping in random coefficient regressions, it is necessary to estimate the distributions of the random coefficients consistently. We show that this is often possible and provide practical representative estimators of these distributions.
Let $(\mathbf{X}_i, Y_i)$ be independent, identically distributed observations that satisfy a logistic regression model; that is, for each $i, \log \lbrack P(Y_i = 1 | \mathbf{X}_i)/P(Y_i = 0 |\mathbf{X}_i)\rbrack = \mathbf{X}^T_i \beta_0$, where $Y_i \in \{0, 1\}, \mathbf{X}_i \in \mathbf{R}^p$ and $\beta_0 \in \mathbf{B}^p$ is the unknown parameter vector of the model. The marginal distribution of the covariate vectors $\mathbf{X}_i$ is assumed to be unknown. Sequential…
We consider the effects of misspecifying the covariate vector on tests of association between a particular covariate and the response variable in logistic regression models. Misspecification can include random measurement error, discretizing a continuous explanatory variable and mismodeling the functional form of an explanatory variable. The effects of misspecification are examined by deriving the asymptotic distribution of the test statistic under a sequence of models.…
We study the asymptotic behavior of Mardia's measure of (sample) multivariate skewness. In the special case of an elliptically symmetric distribution, the limit law is a weighted sum of two independent $\chi^2$-variates. A normal limit distribution arises if the population distribution has positive skewness. These results explain some curiosities in the power performance of a commonly proposed test for multivariate normality based on multivariate skewness.
Orthogonal regression $M$-estimates are considered from a bias robust point of view. Their maximum bias over epsilon-contamination neighborhoods is characterized, and maximum bias curves are computed. The most bias robust orthogonal regression $M$-estimate is derived and shown to be a "mode type" estimate; for instance, in the two-dimensional case this estimate can be computed by locating a strip of fixed width covering the maximum number of data points. It will be shown…
We provide a solution to the smoothing parameter selection problem involved in the construction of adaptive estimates for the symmetric location model and the general linear model. Linear $B$-splines are used to give a simple form of the estimate of the score function of the underlying density. New empirical methods are proposed to locate the knots optimally and to select the number of knots. We also give asymptotic bounds for the empirical selection method and show that an…
Rousseeuw's minimum volume estimator for multivariate location and dispersion parameters has the highest possible breakdown point for an affine equivariant estimator. In this paper we establish that it satisfies a local Holder condition of order $1/2$ and converges weakly at the rate of $n^{-1/3}$ to a non-Gaussian distribution.
In this paper, we outline a general approach to estimating the parametric component of a semiparametric model. For the case of a scalar parametric component, the method is based on the idea of first estimating a one-dimensional subproblem of the original problem that is least favorable in the sense of Stein. The likelihood function for the scalar parameter along this estimated subproblem may be viewed as a generalization of the profile likelihood for the problem. The scalar…
This paper presents a quasi-Bayesian model of subjective uncertainty in which beliefs which are represented by lower and upper probabilities qualified by numerical confidence weights. The representation is derived from a system of axioms of binary preferences which differs from standard axiom systems insofar as completeness is not assumed and transitivity is weakened. Confidence-weighted probabilities may be elicited through the acceptance of bets with limited stakes, a…
As a first step toward developing statistical models based on upper and lower probabilities, we study upper probabilities and upper expectations on the unit interval that are symmetric, by which we mean invariant with respect to equimeasurability. These upper probabilities are generalizations of uniform probability measures. We give some characterizations of these upper probabilities. Specifically, we show that symmetry of the upper expectation functional is equivalent to…
This paper addresses the problem of quantifying expert opinion about a normal linear regression model when there is uncertainty as to which independent variables should be included in the model. Opinion is modeled as a mixture of natural conjugate prior distributions with each distribution in the mixture corresponding to a different subset of the independent variables. It is shown that for certain values of the independent variables, the predictive distribution of the…
A flexible class of prior distributions is proposed, for the covariance matrix of a multivariate normal distribution, yielding much more general hierarchical and empirical Bayes smoothing and inference, when compared with a conjugate analysis involving an inverted Wishart distribution. A likelihood approximation is obtained for the matrix logarithm of the covariance matrix, via Bellman's iterative solution to a Volterra integral equation. Exact and approximate Bayesian,…
In this paper we introduce an appealing nonparametric method for estimating the mean regression function. The proposed method combines the ideas of local linear smoothers and variable bandwidth. Hence, it also inherits the advantages of both approaches. We give expressions for the conditional MSE and MISE of the estimator. Minimization of the MISE leads to an explicit formula for an optimal choice of the variable bandwidth. Moreover, the merits of considering a variable…
In 1989 Kunsch introduced a modified bootstrap and jackknife for a statistic which is used to estimate a parameter of the $m$-dimensional joint distribution of stationary and $\alpha$-mixing observations. The modification amounts to resampling whole blocks of consecutive observations, or deleting whole blocks one at a time. Liu and Singh independently proposed (in 1988) the same technique for observations that are $m$-dependent. However, many time-series statistics, notably…
By providing a probabilistic model for nested case-control sampling in epidemiologic cohort studies, consistency and asymptotic normality of the maximum partial likelihood estimator of regression parameters in a Cox proportional hazards model can be derived using process and martingale theory as in Andersen and Gill. A general expression for the asymptotic variance is given and used to calculate asymptotic relative efficiencies relative to the full cohort variance in some…
In this paper we obtain uniform upper bounds for the $L_1$ error of kernel estimators in estimating monotone densities and densities of bounded variation. The bounds are nonasymptotic and optimal in $n$, the sample size. For the bounded variation class, it is also optimal wrt an upper bound of the total variation. The proofs employ a one-sided kernel technique and are extremely simple.
We consider the problem of bandwidth selection for kernel density estimators. Let $H_n$ denote the bandwidth computed by the least squares cross-validation method. Furthermore, let $H^\ast_n$ and $h^\ast_n$ denote the minimizers of the integrated squared error and the mean integrated squared error, respectively. The main theorem establishes asymptotic normality of $H_n - H^\ast_n$ and $H_n - h^\ast_n$, for three classes of densities with comparable smoothness properties.…
The Robbins-Monro process Xn+1=Xn−cnYn is a standard stochastic approximation method for estimating the root θ of an unknown regression function. There is a vast literature on the convergence properties of Xn to θ. In practice, one is also interested in the conditional distribution of the system under the sequential control when the control is set at θ or near θ. This problem appears to have received no attention in the literature.…
The importance of optimal filtration problem for Markov chain with two states observed in Gaussian white noise (GWN) for a lot of concrete technical problems is well known. The equation for a posterior probability π(t) of one of the states was obtained many years ago. The aim of this paper is to study a simple filtration method. It is shown that this simplified filtration is asymptotically efficient in some sense if the diffusion constant of the GWN goes to 0. Some…
We consider the Akaike-Parzen-Rosenblatt density estimate fnh based upon any superkernel L (i.e., an absolutely integrable function with ∫L=1, whose characteristic function is 1 on [−1,1]), and compare it with a kernel estimate gnh based upon an arbitrary kernel K. We show that for a given subclass of analytic densities, infLsupKlimsupn→∞infhE∫∣gnh−f∣infhE∫∣fnh−f∣=1,…
Let (Xi,Yi) be independent, identically distributed observations that satisfy a logistic regression model; that is, for each i,log[P(Yi=1∣Xi)/P(Yi=0∣Xi)]=XiTβ0, where Yi∈{0,1},Xi∈Rp and β0∈Bp is the unknown parameter vector of the model. The marginal distribution of the covariate vectors Xi is assumed to be unknown. Sequential…
We study the asymptotic behavior of Mardia's measure of (sample) multivariate skewness. In the special case of an elliptically symmetric distribution, the limit law is a weighted sum of two independent χ2-variates. A normal limit distribution arises if the population distribution has positive skewness. These results explain some curiosities in the power performance of a commonly proposed test for multivariate normality based on multivariate skewness.
Orthogonal regression M-estimates are considered from a bias robust point of view. Their maximum bias over epsilon-contamination neighborhoods is characterized, and maximum bias curves are computed. The most bias robust orthogonal regression M-estimate is derived and shown to be a "mode type" estimate; for instance, in the two-dimensional case this estimate can be computed by locating a strip of fixed width covering the maximum number of data points. It will be shown…
We provide a solution to the smoothing parameter selection problem involved in the construction of adaptive estimates for the symmetric location model and the general linear model. Linear B-splines are used to give a simple form of the estimate of the score function of the underlying density. New empirical methods are proposed to locate the knots optimally and to select the number of knots. We also give asymptotic bounds for the empirical selection method and show that an…
Rousseeuw's minimum volume estimator for multivariate location and dispersion parameters has the highest possible breakdown point for an affine equivariant estimator. In this paper we establish that it satisfies a local Holder condition of order 1/2 and converges weakly at the rate of n−1/3 to a non-Gaussian distribution.
A General Resampling Scheme for Triangular Arrays of α-Mixing Random Variables with Application to the Problem of Spectral Density Estimation
In 1989 Kunsch introduced a modified bootstrap and jackknife for a statistic which is used to estimate a parameter of the m-dimensional joint distribution of stationary and α-mixing observations. The modification amounts to resampling whole blocks of consecutive observations, or deleting whole blocks one at a time. Liu and Singh independently proposed (in 1988) the same technique for observations that are m-dependent. However, many time-series statistics, notably…
Some Nonasymptotic Bounds for L1 Density Estimation using Kernels
In this paper we obtain uniform upper bounds for the L1 error of kernel estimators in estimating monotone densities and densities of bounded variation. The bounds are nonasymptotic and optimal in n, the sample size. For the bounded variation class, it is also optimal wrt an upper bound of the total variation. The proofs employ a one-sided kernel technique and are extremely simple.
We consider the problem of bandwidth selection for kernel density estimators. Let Hn denote the bandwidth computed by the least squares cross-validation method. Furthermore, let Hn∗ and hn∗ denote the minimizers of the integrated squared error and the mean integrated squared error, respectively. The main theorem establishes asymptotic normality of Hn−Hn∗ and Hn−hn∗, for three classes of densities with comparable smoothness properties.…