Some standard numerical problems become intractable in high dimensions. Yet successes are often achieved in practice. This may be explained in terms of the underlying target function being somehow simpler than assumed in the intractability arguments. A prototypical form of simplicity is approximate linearity. In moderate dimensions, linearity can be investigated by linear regression. In very high dimensions, this becomes computationally inefficient and eventually infeasible,…
We propose a method of inference for generalized linear mixed models (GLMM) that in many ways resembles the method of least squares. We also show that adequate inference about GLMM can be made based on the conditional likelihood on a subset of the random effects. One of the important features of our methods is that they rely on weak distributional assumptions about the random effects. The methods proposed are also computationally feasible. Asymptotic behavior of the…
Deng and Tang proposed generalized resolution and minimum aberration criteria for comparing and assessing nonregular fractional factorials, of which Plackett–Burman designs are special cases.A relaxed variant of generalized aberration is proposed and studied in this paper.We show that a best design according to this criterion minimizes the contamination of nonnegligible interactions on the estimation of main effects in the order of importance given by the hierarchical…
Kokyo Choy · Marc Hallin · Abdeslam Serroukh · Masanobu Taniguchi·刊期 1999-12-01 · 卷 27（6）·The Annals of Statistics已编入正刊
长记忆扰动回归模型的局部渐近正态性AI 解读
The local asymptotic normality property is established for a regression model with fractional ARIMA($p, d, q$) errors. This result allows for solving, in an asymptotically optimal way, a variety of inference problems in the long-memory context: hypothesis testing, discriminant analysis, rank-based testing, locally asymptotically minimax andadaptive estimation, etc. The problem of testing linear constraints on the parameters, the discriminant analysis problem, and the…
The problem of estimating a density $g$ based on a sample $X_1, X_2,\dots, X_n$ from $p = q*g$ is considered. Linear and nonlinear wavelet estimators based on Meyer-type wavelets are constructed. The estimators are asymptotically optimal and adaptive if $g$ belongs to the Sobolev space $H^{\alpha}$ . Moreover, the estimators considered in this paper adjust automatically to the situation when $g$ is supersmooth.
In this paper,we consider a multivariate one-way random effect model with equal replications. We propose nonnegative definite estimators for “between” and “within” components of variance. Under the Stein loss function, it is shown that the proposed estimators of the “within” component dominate the best unbiased estimator. Restricted maximum likelihood, truncated and order-preserving minimax estimators are also proposed. A Monte Carlo simulation is carried out to choose among…
In this paper, we define the blocking wordlength pattern of a blocked fractional factorial design by combining the wordlength patterns of treatment-defining words and block-defining words. The concept of minimum aberration can be defined in terms of the blocking wordlength pattern and provides a good measure of the estimation capacity of a blocked fractional factorial design. By blending techniques of coding theory and finite projective geometry, we obtain combinatorial…
Abba M. Krieger · Moshe Pollak · Benjamin Yakir·刊期 1999-12-01 · 卷 27（6）·The Annals of Statistics已编入正刊
回归变化检测的一阶最优性AI 解读
Observations are generated according to a regression with normal error as a function of time,when the process is in control. The process potentially changes at some unknown point oftime and then the ensuing observations are normal with the same mean function plus an arbitrary function under suitable regularity conditions. The problem is to obtain a stopping rule that is optimal in the sense that the rule minimizes the expected delay in detecting a change subject to a…
Sequential randomized prediction of an arbitrary binary sequence is investigated. No assumption is made on the mechanism of generating the bit sequence. The goal of the predictor is to minimize its relative loss (or regret), that is, to make almost as few mistakes as the best “expert” in a fixed, possibly infinite, set of experts. We point out a surprising connection between this prediction problem and empirical process theory. First, in the special case of static…
Given $n$ independent replicates of a jointly distributed pair $(X, Y) \in \mathscr{R}^d \times \mathscr{R}$, we wish to select from a fixed sequence of model classes $\mathscr{F}_1, \mathscr{F}_2,\dots$ a deterministic prediction rule $f: \mathscr{R}^d \to \mathscr{R}$ whose risk is small.We investigate the possibility of empirically assessing the complexity of each model class, that is, the actual difficulty of the estimation problem within each class. The estimated…
Discriminant analysis for two data sets in $\mathbb{R}^d$ with probability densities $f$ and $g$ can be based on the estimation of the set $G = \{x: f(x) \geq g(x)\}$. We consider applications where it is appropriate to assume that the region $G$ has a smooth boundary or belongs to another nonparametric class of sets. In particular, this assumption makes sense if discrimination is used as a data analytic tool. Decision rules based on minimization of empirical risk over the…
In this paper, a nonparametric Bayesian analysis of queueing models with geometric input and general service time is performed. In particular, statistical inference for the probability generating function p.g.f. of the equilibrium waiting time distribution is considered. The consistency of the posterior distribution for such a p.g.f., as well as the weak convergence to a Gaussian process of a suitable rescaling, are proved. As by-products, results on statistical inference…
Consider a heteroscedastic regression model $Y = m(X) + \sigma(X)\varepsilon$, where the functions $m$ and $\sigma$ are “smooth,” and $\varepsilon$ is independent of $X$. The response variable $Y$ is subject to random censoring, but it is assumed that there exists a region of the covariate $X$ where the censoring of $Y$ is “light.” Under this condition, it is shown that the assumed nonparametric regression model can be used to transfer tail information from regions of light…
We derive the asymptotic distribution ofthe maximal depth regression estimator recently proposed in Rousseeuw and Hubert. The estimator is obtained by maximizing a projection-based depth and the limiting distribution is characterized through a max–min operation of a continuous process. The same techniques can be used to obtain the limiting distribution of some other depth estimators including Tukey’s deepest point based on half-space depth. Results for the special case of…
We study maximum penalized likelihood density estimation using the first roughness penalty functional of Good. We prove a simple pointwise comparison result with a kernel estimator based on the two-sided exponential kernel. This leads to $L^1$ convergence results similar to those for kernel estimators. We also prove Hellinger distance bounds for the roughness penalized estimator.
We derive the asymptotic distribution of a new backfitting procedure for estimating the closest additive approximation to a nonparametric regression function. The procedure employs a recent projection interpretation of popular kernel estimators provided by Mammen, Marron, Turlach and Wand and the asymptotic theory of our estimators is derived using the theory of additive projections reviewed in Bickel, Klaassen, Ritov and Wellner. Our procedure achieves the same bias and…
Many of the popular nonparametric test statistics for censored survival data used in two-sample, $k$-sample trend and continuous covariate situations are special cases of a general statistic, differing only in the choice of the covariate-based label and the weight function. A weight function determines the asymptotic efficiency of its corresponding statistic in this general class. Since the true alternatives are often unknown, we may not be able to foresee which weight…
Given a mixture of binomial distributions, how do we estimate the unknown mixing distribution? We build on earlier work of Lindsay and further elucidate the geometry underlying this question, exploring the approximating role played by cyclic polytopes. Convergence of a resulting maximum likelihood fitting algorithm is proved and numerical examples given; problems over the lack of identifiability of the mixing distribution in part disappear.
In a recent paper, we proposed a new estimation method for the blind deconvolution of a linear system with discrete random input, when the observations may be noise perturbed. We give here asymptotic properties of the estimators in the parametric situation. With nonnoisy observations, the speed of convergence is governed by the $l_1$-tail of the inverse filter, which may have an exponential decrease. With noisy observations, the estimator satisfies a limit theorem with known…
Rao, Pathak and Koltchinskii have recently studied a sequential approach to resampling in which resampling is carried out sequentially one-by-one (with replacement each time) until the bootstrap sample contains $m \approx (1 - e^{-1})n \approx 0.632n$ distinct observations from the original sample. In our previous work, we have established that the main empirical characteristics of the sequential bootstrap go through, in the sense of being within a distance $O(n^{-3/4})$ from…
We investigate the asymptotic behavior of a weighted sample mean and covariance,where the weights are determined by the Mahalanobis distances with respect to initial robust estimators.We derive an explicit expansion for the weighted estimators. From this expansion it can be seen that reweighting does not improve the rate of convergence of the initial estimators.We also show that if one uses smooth $S$-estimators to determine the weights, the weighted estimators are…
The partly linear additive Cox model is an extension of the (linear) Cox model and allows flexible modeling of covariate effects semiparametrically. We study asymptotic properties of the maximum partial likelihood estimator of this model with right-censored data using polynomial splines. We show that, with a range of choices of the smoothing parameter (the number of spline basis functions) required for estimation of the nonparametric components, the estimator of the…
Varying coefficient models are a useful extension of classical linear models. They arise naturally when one wishes to examine how regression coefficients change over different groups characterized by certain covariates such as age. The appeal of these models is that the coef .cient functions can easily be estimated via a simple local regression.This yields a simple one-step estimation procedure. We show that such a one-step method cannot be optimal when different coefficient…
We present some general results determining minimax bounds on statistical risk for density estimation based on certain information-theoretic considerations. These bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence.
Minimum G2-aberration for nonregular fractional factorial designs
非正规部分因子设计的最小 G2 混杂准则
The local asymptotic normality property is established for a regression model with fractional ARIMA(p,d,q) errors. This result allows for solving, in an asymptotically optimal way, a variety of inference problems in the long-memory context: hypothesis testing, discriminant analysis, rank-based testing, locally asymptotically minimax andadaptive estimation, etc. The problem of testing linear constraints on the parameters, the discriminant analysis problem, and the…
The problem of estimating a density g based on a sample X1,X2,…,Xn from p=q∗g is considered. Linear and nonlinear wavelet estimators based on Meyer-type wavelets are constructed. The estimators are asymptotically optimal and adaptive if g belongs to the Sobolev space Hα . Moreover, the estimators considered in this paper adjust automatically to the situation when g is supersmooth.
Theory of optimal blocking of 2n−m designs
Given n independent replicates of a jointly distributed pair (X,Y)∈Rd×R, we wish to select from a fixed sequence of model classes F1,F2,… a deterministic prediction rule f:Rd→R whose risk is small.We investigate the possibility of empirically assessing the complexity of each model class, that is, the actual difficulty of the estimation problem within each class. The estimated…
Discriminant analysis for two data sets in Rd with probability densities f and g can be based on the estimation of the set G={x:f(x)≥g(x)}. We consider applications where it is appropriate to assume that the region G has a smooth boundary or belongs to another nonparametric class of sets. In particular, this assumption makes sense if discrimination is used as a data analytic tool. Decision rules based on minimization of empirical risk over the…
Large sample Bayesian analysis for Geo/G/1 discrete-time queueing models
Consider a heteroscedastic regression model Y=m(X)+σ(X)ε, where the functions m and σ are “smooth,” and ε is independent of X. The response variable Y is subject to random censoring, but it is assumed that there exists a region of the covariate X where the censoring of Y is “light.” Under this condition, it is shown that the assumed nonparametric regression model can be used to transfer tail information from regions of light…
We study maximum penalized likelihood density estimation using the first roughness penalty functional of Good. We prove a simple pointwise comparison result with a kernel estimator based on the two-sided exponential kernel. This leads to L1 convergence results similar to those for kernel estimators. We also prove Hellinger distance bounds for the roughness penalized estimator.
Many of the popular nonparametric test statistics for censored survival data used in two-sample, k-sample trend and continuous covariate situations are special cases of a general statistic, differing only in the choice of the covariate-based label and the weight function. A weight function determines the asymptotic efficiency of its corresponding statistic in this general class. Since the true alternatives are often unknown, we may not be able to foresee which weight…
In a recent paper, we proposed a new estimation method for the blind deconvolution of a linear system with discrete random input, when the observations may be noise perturbed. We give here asymptotic properties of the estimators in the parametric situation. With nonnoisy observations, the speed of convergence is governed by the l1-tail of the inverse filter, which may have an exponential decrease. With noisy observations, the estimator satisfies a limit theorem with known…
Rao, Pathak and Koltchinskii have recently studied a sequential approach to resampling in which resampling is carried out sequentially one-by-one (with replacement each time) until the bootstrap sample contains m≈(1−e−1)n≈0.632n distinct observations from the original sample. In our previous work, we have established that the main empirical characteristics of the sequential bootstrap go through, in the sense of being within a distance O(n−3/4) from…
We investigate the asymptotic behavior of a weighted sample mean and covariance,where the weights are determined by the Mahalanobis distances with respect to initial robust estimators.We derive an explicit expansion for the weighted estimators. From this expansion it can be seen that reweighting does not improve the rate of convergence of the initial estimators.We also show that if one uses smooth S-estimators to determine the weights, the weighted estimators are…