论文检索

GAP：两样本稀疏推断的信息汇聚通用框架AI 解读

Yin Xiaa, T. Tony Caib & Wenguang Sunc*a Department of Statistics, School of Management, Fudan University, Shanghai, China; b Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA; c Department of Data Sciences and Operations, University of Southern California, Los Angeles, CAColor versions of one or more of the figures in the article can be found online at www.tandfonline.com/r/JASA.Supplementary materials for this article are available…

稀疏矩阵检测的统计与计算极限AI 解读

This paper investigates the fundamental limits for detecting a high-dimensional sparse matrix contaminated by white Gaussian noise from both the statistical and computational perspectives. We consider $p\times p$ matrices whose rows and columns are individually $k$-sparse. We provide a tight characterization of the statistical and computational limits for sparse matrix detection, which precisely describe when achieving optimal detection is easy, hard or impossible,…

发散尖峰特征值与最大非尖峰样本协方差特征值的极限规律AI 解读

We study the asymptotic distributions of the spiked eigenvalues and the largest nonspiked eigenvalue of the sample covariance matrix under a general covariance model with divergent spiked eigenvalues, while the other eigenvalues are bounded but otherwise arbitrary. The limiting normal distribution for the spiked sample eigenvalues is established. It has distinct features that the asymptotic mean relies on not only the population spikes but also the nonspikes and that the…

高维线性回归解释方差的半监督推断及其应用AI 解读

Summary The paper considers statistical inference for the explained variance βTΣβ under the high dimensional linear model Y = Xβ + ε in the semisupervised setting, where β is the regression vector and Σ is the design covariance matrix. A calibrated estimator, which efficiently integrates both labelled and unlabelled data, is proposed. It is shown that the estimator achieves the minimax optimal rate of convergence in the general semisupervised framework. The optimality result…

基于高频数据的高维最小方差组合估计AI 解读

This paper studies the estimation of high-dimensional minimum variance portfolio (MVP) based on the high frequency returns which can exhibit heteroscedasticity and possibly be contaminated by microstructure noise. Under certain sparsity assumptions on the precision matrix, we propose estimators of the MVP and prove that our portfolios asymptotically achieve the minimum variance in a sharp sense. In addition, we introduce consistent estimators of the minimum variance, which…

半监督推断：一般理论与总体均值估计AI 解读

We propose a general semi-supervised inference framework focused on the estimation of the population mean. As usual in semi-supervised settings, there exists an unlabeled sample of covariate vectors and a labeled sample consisting of covariate vectors along with real-valued responses (“labels”). Otherwise, the formulation is “assumption-lean” in that no major conditions are imposed on the statistical or functional form of the data. We consider both the ideal semi-supervised…

高维线性判别分析：最优性、自适应算法与缺失数据AI 解读

Summary The paper develops optimality theory for linear discriminant analysis in the high dimensional setting. A data-driven and tuning-free classification rule, which is based on an adaptive constrained l1-minimization approach, is proposed and analysed. Minimax lower bounds are obtained and this classification rule is shown to be simultaneously rate optimal over a collection of parameter spaces. In addition, we consider classification with incomplete data under the…

多组大规模多重检验整合分析中的信号分类AI 解读

Summary The integrative analysis of multiple data sets is becoming increasingly important in many fields of research. When the same features are studied in several independent experiments, it can often be useful to analyse jointly the multiple sequences of multiple tests that result. It is frequently necessary to classify each feature into one of several categories, depending on the null and non-null configuration of its corresponding test statistics. The paper studies this…

CHIME：基于EM算法的高维高斯混合聚类及其最优性AI 解读

Unsupervised learning is an important problem in statistics and machine learning with a wide range of applications. In this paper, we study clustering of high-dimensional Gaussian mixtures and propose a procedure, called CHIME, that is based on the EM algorithm and a direct estimation method for the sparse discriminant vector. Both theoretical and numerical properties of CHIME are investigated. We establish the optimal rate of convergence for the excess misclustering error…

大规模两样本推断中的协变量辅助排序与筛选AI 解读

Summary Two-sample multiple testing has a wide range of applications. The conventional practice first reduces the original observations to a vector of p-values and then chooses a cut-off to adjust for multiplicity. However, this data reduction step could cause significant loss of information and thus lead to suboptimal testing procedures. We introduce a new framework for two-sample multiple testing by incorporating a carefully constructed auxiliary variable in inference to…

高维线性模型中遗传关联性的最优估计AI 解读

Estimating the genetic relatedness between two traits based on the genome-wide association data is an important problem in genetics research. In the framework of high-dimensional linear models, we introduce two measures of genetic relatedness and develop optimal estimators for them. One is genetic covariance, which is defined to be the inner product of the two regression vectors, and another is genetic correlation, which is a normalized inner product by their lengths. We…

高维协变量下的内生性检验AI 解读

Modern, high dimensional data has renewed investigation on instrumental variables (IV) analysis, primarily focusing on estimation of effects of endogenous variables and putting little attention towards specification tests. This paper studies in high dimensions the Durbin–Wu–Hausman (DWH) test, a popular specification test for endogeneity in IV regression. We show, surprisingly, that the DWH test maintains its size in high dimensions, but at an expense of power. We propose a…

存在无效工具变量时的因果效应置信区间：两阶段硬阈值与投票AI 解读

Summary A major challenge in instrumental variable (IV) analysis is to find instruments that are valid, or have no direct effect on the outcome and are ignorable. Typically one is unsure whether all of the putative IVs are in fact valid. We propose a general inference procedure in the presence of invalid IVs, called two-stage hard thresholding with voting. The procedure uses two hard thresholding steps to select strong instruments and to generate candidate sets of valid IVs.…

高维线性回归估计的精度评估AI 解读

This paper considers point and interval estimation of the $\ell_{q}$ loss of an estimator in high-dimensional linear regression with random design. We establish the minimax rate for estimating the $\ell_{q}$ loss and the minimax expected length of confidence intervals for the $\ell_{q}$ loss of rate-optimal estimators of the regression vector, including commonly used estimators such as Lasso, scaled Lasso, square-root Lasso and Dantzig Selector. Adaptivity of confidence…

大规模多重检验中的加权错误发现率控制AI 解读

The use of weights provides an effective strategy to incorporate prior domain knowledge in large-scale inference. This paper studies weighted multiple testing in a decision-theoretic framework. We develop oracle and data-driven procedures that aim to maximize the expected number of true positives subject to a constraint on the weighted false discovery rate. The asymptotic validity and optimality of the proposed methods are established. The results demonstrate that…

平面凸集的自适应估计AI 解读

In this paper, we consider adaptive estimation of an unknown planar compact, convex set from noisy measurements of its support function. Both the problem of estimating the support function at a point and that of estimating the whole convex set are studied. For pointwise estimation, we consider the problem in a general nonasymptotic framework, which evaluates the performance of a procedure at each individual set, instead of the worst case performance over a large parameter…

奇异子空间的速率最优扰动界及高维统计应用AI 解读

Perturbation bounds for singular spaces, in particular Wedin’s $\mathop{\mathrm{sin}}\nolimits \Theta$ theorem, are a fundamental tool in many fields including high-dimensional statistics, machine learning and applied mathematics. In this paper, we establish separate perturbation bounds, measured in both spectral and Frobenius $\mathop{\mathrm{sin}}\nolimits \Theta$ distances, for the left and right singular subspaces. Lower bounds, which show that the individual…

大型噪声矩阵中子矩阵定位的计算与统计边界AI 解读

We study in this paper computational and statistical boundaries for submatrix localization. Given one observation of (one or multiple nonoverlapping) signal submatrix (of magnitude $\lambda$ and size $k_{m}\times k_{n}$) embedded in a large noise matrix (of size $m\times n$), the goal is to optimal identify the support of the signal submatrix computationally and statistically. Two transition thresholds for the signal-to-noise ratio $\lambda/\sigma$ are established in terms…

用于识别遗传调控疾病基因的稀疏同时信号检测AI 解读

Genome-wide association studies (GWAS) and differential expression analyses have had limited success in finding genes that cause complex diseases such as heart failure (HF), a leading cause of death in the United States. This article proposes a new statistical approach that integrates GWAS and expression quantitative trait loci (eQTL) data to identify important HF genes. For such genes, genetic variations that perturb its expression are also likely to influence disease risk.…

高维线性回归置信区间的极小极大速率与自适应性AI 解读

Confidence sets play a fundamental role in statistical inference. In this paper, we consider confidence intervals for high-dimensional linear regression with random design. We first establish the convergence rates of the minimax expected length for confidence intervals in the oracle setting where the sparsity parameter is given. The focus is then on the problem of adaptation to sparsity for the construction of confidence intervals. Ideally, an adaptive confidence interval…

稀疏信号的最优筛选与发现：多阶段高通量研究应用AI 解读

Summary A common feature in large-scale scientific studies is that signals are sparse and it is desirable to narrow down significantly the focus to a much smaller subset in a sequential manner. We consider two related data screening problems: one is to find the smallest subset such that it virtually contains all signals and another is to find the largest subset such that it essentially contains only signals. These screening problems are closely connected to but distinct from…

对《高维聚类中的重要特征主成分分析》的讨论AI 解读

We would like to congratulate the authors for an interesting paper and a novel proposal for clustering high-dimensional Gaussian mixtures with a diagonal covariance matrix.The proposed two-stage procedure first selects features based on the Kolmogorov-Smirnov statistics and then applies a spectral clustering method to the post-selected data.A rigorous theoretical analysis for the clustering error is given and the results are supported by a competitive performance in…

阈值化 Wirtinger 流实现含噪稀疏相位恢复的最优收敛速率AI 解读

This paper considers the noisy sparse phase retrieval problem: recovering a sparse signal $\mathbf{x}\in\mathbb{R}^{p}$ from noisy quadratic measurements $y_{j}=(\mathbf{a}_{j}'\mathbf{x})^{2}+\varepsilon_{j}$, $j=1,\ldots,m$, with independent sub-exponential noise $\varepsilon_{j}$. The goals are to understand the effect of the sparsity of $\mathbf{x}$ on the estimation precision and to construct a computationally feasible estimator to achieve the optimal rates adaptively.…

一般高维线性逆问题的几何推断AI 解读

This paper presents a unified geometric framework for the statistical analysis of a general ill-posed linear inverse model which includes as special cases noisy compressed sensing, sign vector recovery, trace regression, orthogonal matrix estimation and noisy matrix completion. We propose computationally feasible convex programs for statistical inference including estimation, confidence intervals and hypothesis testing. A theoretical framework is developed to characterize…