The problem of structure estimation in graphical models with latent variables is considered. We characterize conditions for tractable graph estimation and develop efficient methods with provable guarantees. We consider models where the underlying Markov graph is locally tree-like, and the model is in the regime of correlation decay. For the special case of the Ising model, the number of samples $n$ required for structural consistency of our method scales as $n=\Omega(\theta_{\min}^{-\delta\eta(\eta+1)-2}\log p)$…
Given independent samples from $P$ and $Q$, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is $P=Q$. On the other hand, when comparing or testing particular parameters $\theta$ of $P$ and $Q$, such as their means or medians, permutation tests need not be level $\alpha$, or even approximately level $\alpha$ in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the…
It is shown that the fiducial distribution in a group model, or more generally a quasigroup model, determines the optimal equivariant frequentist inference procedures. The proof does not rely on existence of invariant measures, and generalizes results corresponding to the choice of the right Haar measure as a Bayesian prior. Classical and more recent examples show that fiducial arguments can be used to give good candidates for exact or approximate confidence distributions.…
A Cramér moderate deviation theorem for Hotelling’s $T^{2}$-statistic is proved under a finite $(3+\delta)$th moment. The result is applied to large scale tests on the equality of mean vectors and is shown that the number of tests can be as large as $e^{o(n^{1/3})}$ before the chi-squared distribution calibration becomes inaccurate. As an application of the moderate deviation results, a global test on the equality of $m$ mean vectors based on the maximum of Hotelling’s $T^{2}$…
We develop asymptotic theory for weighted likelihood estimators (WLE) under two-phase stratified sampling without replacement. We also consider several variants of WLEs involving estimated weights and calibration. A set of empirical process tools are developed including a Glivenko–Cantelli theorem, a theorem for rates of convergence of $M$-estimators, and a Donsker theorem for the inverse probability weighted empirical processes under two-phase sampling and sampling without…
We develop an efficient estimation procedure for identifying and estimating the central subspace. Using a new way of parameterization, we convert the problem of identifying the central subspace to the problem of estimating a finite dimensional parameter in a semiparametric model. This conversion allows us to derive an efficient estimator which reaches the optimal semiparametric efficiency bound. The resulting efficient estimator can exhaustively estimate the central subspace…
The causal inference literature has provided a clear formal definition of confounding expressed in terms of counterfactual independence. The literature has not, however, come to any consensus on a formal definition of a confounder, as it has given priority to the concept of confounding over that of a confounder. We consider a number of candidate definitions arising from various more informal statements made in the literature. We consider the properties satisfied by each…
Xuming He · Lan Wang · Hyokyoung Grace Hong·刊期 2013-02-01 · 卷 41(1)·The Annals of Statistics已编入正刊
高维异质数据的分位数自适应无模型变量筛选AI 解读
We introduce a quantile-adaptive framework for nonlinear variable screening with high-dimensional heterogeneous data. This framework has two distinctive features: (1) it allows the set of active variables to vary across quantiles, thus making it more flexible to accommodate heterogeneity; (2) it is model-free and avoids the difficult task of specifying the form of a statistical model in a high dimensional space. Our nonlinear independence screening procedure employs spline…
Kuang-Yao Lee · Bing Li · Francesca Chiaromonte·刊期 2013-02-01 · 卷 41(1)·The Annals of Statistics已编入正刊
非线性充分降维的一般理论:表述与估计AI 解读
In this paper we introduce a general theory for nonlinear sufficient dimension reduction, and explore its ramifications and scope. This theory subsumes recent work employing reproducing kernel Hilbert spaces, and reveals many parallels between linear and nonlinear sufficient dimension reduction. Using these parallels we analyze the properties of existing methods and develop new ones. We begin by characterizing dimension reduction at the general level of $\sigma$-fields and…
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and $f$-divergence functionals such as Hellinger and Kullback–Leibler distances on the space of mixture distributions is investigated in detail using various identifiability conditions. Convergence in Wasserstein metrics for…
This article presents an algorithm that generates a conservative confidence interval of a specified length and coverage probability for the power of a Monte Carlo test (such as a bootstrap or permutation test). It is the first method that achieves this aim for almost any Monte Carlo test. Previous research has focused on obtaining as accurate a result as possible for a fixed computational effort, without providing a guaranteed precision in the above sense. The algorithm we…
We analyze the convergence rate of a simplified version of a popular Gibbs sampling method used for statistical discovery of gene regulatory binding motifs in DNA sequences. This sampler satisfies a very strong form of ergodicity (uniform). However, we show that, due to multimodality of the posterior distribution, the rate of convergence often decreases exponentially as a function of the length of the DNA sequence. Specifically, we show that this occurs whenever there is…
While adaptive sensing has provided improved rates of convergence in sparse regression and classification, results in nonparametric regression have so far been restricted to quite specific classes of functions. In this paper, we describe an adaptive-sensing algorithm which is applicable to general nonparametric-regression problems. The algorithm is spatially adaptive, and achieves improved rates of convergence over spatially inhomogeneous functions. Over standard function…
Researchers are often interested in drawing inferences regarding the order between two experimental groups on the basis of multivariate response data. Since standard multivariate methods are designed for two sided alternatives they may not be ideal for testing for order between two groups. In this article we introduce the notion of the linear stochastic order and investigate its properties. Statistical theory and methodology are developed to both estimate the direction which…
We consider the problem of estimating the mean vector of a $p$-variate normal $(\theta,\Sigma)$ distribution under invariant quadratic loss, $(\delta-\theta)'\Sigma^{-1}(\delta-\theta)$, when the covariance is unknown. We propose a new class of estimators that dominate the usual estimator $\delta^{0}(X)=X$. The proposed estimators of $\theta$ depend upon $X$ and an independent Wishart matrix $S$ with $n$ degrees of freedom, however, $S$ is singular almost surely when $p>n$.…
This paper studies estimation in functional linear quantile regression in which the dependent variable is scalar while the covariate is a function, and the conditional quantile for each fixed quantile index is modeled as a linear functional of the covariate. Here we suppose that covariates are discretely observed and sampling points may differ across subjects, where the number of measurements per subject increases as the sample size. Also, we allow the quantile index to vary…
Anatoli Juditsky · Fatma Kılınç Karzan · Arkadi Nemirovski · Boris Polyak·刊期 2012-12-01 · 卷 40(6)·The Annals of Statistics已编入正刊
块稀疏信号 ℓ₁ 恢复的精度保证AI 解读
We introduce a general framework to handle structured models (sparse and block-sparse with possibly overlapping blocks). We discuss new methods for their recovery from incomplete observation, corrupted with deterministic and stochastic noise, using block-$\ell_{1}$ regularization. While the current theory provides promising bounds for the recovery errors under a number of different, yet mostly hard to verify conditions, our emphasis is on verifiable conditions on the problem…
A random-walk Metropolis sampler is geometrically ergodic if its equilibrium density is super-exponentially light and satisfies a curvature condition [Stochastic Process. Appl. 85 (2000) 341–361]. Many applications, including Bayesian analysis with conjugate priors of logistic and Poisson regression and of log-linear models for categorical data result in posterior distributions that are not super-exponentially light. We show how to apply the change-of-variable formula for…
Although the leave-subject-out cross-validation (CV) has been widely used in practice for tuning parameter selection for various nonparametric and semiparametric models of longitudinal data, its theoretical property is unknown and solving the associated optimization problem is computationally expensive, especially when there are multiple tuning parameters. In this paper, by focusing on the penalized spline method, we show that the leave-subject-out CV is optimal in the sense…
Hira L. Koul · Ursula U. Müller · Anton Schick·刊期 2012-12-01 · 卷 40(6)·The Annals of Statistics已编入正刊
转移原理:完整案例分析的工具AI 解读
This paper gives a general method for deriving limiting distributions of complete case statistics for missing data models from corresponding results for the model where all data are observed. This provides a convenient tool for obtaining the asymptotic behavior of complete case versions of established full data methods without lengthy proofs. The methodology is illustrated by analyzing three inference procedures for partially linear regression models with responses missing…
Independent Component Analysis (ICA) models are very popular semiparametric models in which we observe independent copies of a random vector $X=AS$, where $A$ is a non-singular matrix and $S$ has independent components. We propose a new way of estimating the unmixing matrix $W=A^{-1}$ and the marginal distributions of the components of $S$ using nonparametric maximum likelihood. Specifically, we study the projection of the empirical distribution onto the subset of ICA…
We propose a two-step estimating procedure for generalized additive partially linear models with clustered data using estimating equations. Our proposed method applies to the case that the number of observations per cluster is allowed to increase with the number of independent subjects. We establish oracle properties for the two-step estimator of each function component such that it performs as well as the univariate function estimator by assuming that the parametric vector…
Robert J. Adler · Eliran Subag · Jonathan E. Taylor·刊期 2012-12-01 · 卷 40(6)·The Annals of Statistics已编入正刊
旋转与尺度空间随机场及高斯运动学公式AI 解读
We provide a new approach, along with extensions, to results in two important papers of Worsley, Siegmund and coworkers closely tied to the statistical analysis of fMRI (functional magnetic resonance imaging) brain data. These papers studied approximations for the exceedence probabilities of scale and rotation space random fields, the latter playing an important role in the statistical analysis of fMRI data. The techniques used there came either from the Euler characteristic…
Eduard Belitser · Subhashis Ghosal · Harry van Zanten·刊期 2012-12-01 · 卷 40(6)·The Annals of Statistics已编入正刊
估计多元回归函数最大值位置与大小的最优两阶段程序AI 解读
We propose a two-stage procedure for estimating the location $\mathbf{\mu}$ and size $M$ of the maximum of a smooth $d$-variate regression function $f$. In the first stage, a preliminary estimator of $\mathbf{\mu}$ obtained from a standard nonparametric smoothing method is used. At the second stage, we “zoom-in” near the vicinity of the preliminary estimator and make further observations at some design points in that vicinity. We fit an appropriate polynomial regression…
The problem of structure estimation in graphical models with latent variables is considered. We characterize conditions for tractable graph estimation and develop efficient methods with provable guarantees. We consider models where the underlying Markov graph is locally tree-like, and the model is in the regime of correlation decay. For the special case of the Ising model, the number of samples n required for structural consistency of our method scales as n=Ω(θmin−δη(η+1)−2logp)…
Given independent samples from P and Q, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is P=Q. On the other hand, when comparing or testing particular parameters θ of P and Q, such as their means or medians, permutation tests need not be level α, or even approximately level α in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the…
A Cramér moderate deviation theorem for Hotelling’s T2-statistic with applications to global tests
A Cramér moderate deviation theorem for Hotelling’s T2-statistic is proved under a finite (3+δ)th moment. The result is applied to large scale tests on the equality of mean vectors and is shown that the number of tests can be as large as eo(n1/3) before the chi-squared distribution calibration becomes inaccurate. As an application of the moderate deviation results, a global test on the equality of m mean vectors based on the maximum of Hotelling’s T2…
We develop asymptotic theory for weighted likelihood estimators (WLE) under two-phase stratified sampling without replacement. We also consider several variants of WLEs involving estimated weights and calibration. A set of empirical process tools are developed including a Glivenko–Cantelli theorem, a theorem for rates of convergence of M-estimators, and a Donsker theorem for the inverse probability weighted empirical processes under two-phase sampling and sampling without…
In this paper we introduce a general theory for nonlinear sufficient dimension reduction, and explore its ramifications and scope. This theory subsumes recent work employing reproducing kernel Hilbert spaces, and reveals many parallels between linear and nonlinear sufficient dimension reduction. Using these parallels we analyze the properties of existing methods and develop new ones. We begin by characterizing dimension reduction at the general level of σ-fields and…
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f-divergence functionals such as Hellinger and Kullback–Leibler distances on the space of mixture distributions is investigated in detail using various identifiability conditions. Convergence in Wasserstein metrics for…
Improved multivariate normal mean estimation with unknown covariance when p is greater than n
We consider the problem of estimating the mean vector of a p-variate normal (θ,Σ) distribution under invariant quadratic loss, (δ−θ)′Σ−1(δ−θ), when the covariance is unknown. We propose a new class of estimators that dominate the usual estimator δ0(X)=X. The proposed estimators of θ depend upon X and an independent Wishart matrix S with n degrees of freedom, however, S is singular almost surely when p>n.…
Accuracy guaranties for ℓ1 recovery of block-sparse signals
We introduce a general framework to handle structured models (sparse and block-sparse with possibly overlapping blocks). We discuss new methods for their recovery from incomplete observation, corrupted with deterministic and stochastic noise, using block-ℓ1 regularization. While the current theory provides promising bounds for the recovery errors under a number of different, yet mostly hard to verify conditions, our emphasis is on verifiable conditions on the problem…
Independent Component Analysis (ICA) models are very popular semiparametric models in which we observe independent copies of a random vector X=AS, where A is a non-singular matrix and S has independent components. We propose a new way of estimating the unmixing matrix W=A−1 and the marginal distributions of the components of S using nonparametric maximum likelihood. Specifically, we study the projection of the empirical distribution onto the subset of ICA…
We propose a two-stage procedure for estimating the location μ and size M of the maximum of a smooth d-variate regression function f. In the first stage, a preliminary estimator of μ obtained from a standard nonparametric smoothing method is used. At the second stage, we “zoom-in” near the vicinity of the preliminary estimator and make further observations at some design points in that vicinity. We fit an appropriate polynomial regression…