论文检索

有向网络互惠效应的最优估计与检验AI 解读

The $p_1$ model plays a fundamental role in modeling directed networks, where the reciprocal effect parameter $ρ$ is of special interest in practice. However, due to nonlinear factors in this model, how to estimate $ρ$ efficiently is a long-standing open problem. We tackle the problem by the cycle count approach. The challenge is, due to the nonlinear factors in the model, for any given type of generalized cycles, the expected count is a complicated function of many…

块模型家族的网络拟合优度检验AI 解读

The block-model family has four popular network models (SBM, DCBM, MMSBM, and DCMM). A fundamental problem is, how well each of these models fits with real networks. We propose GoF-MSCORE as a new Goodness-of-Fit (GoF) metric for DCMM (the broadest one among the four), with two main ideas. The first is to use cycle count statistics as a general recipe for GoF. The second is a novel network fitting scheme. GoF-MSCORE is a flexible GoF approach, and we further extend it to…

网络两样本比较的最优检验AI 解读

We are interested in the problem of two-sample network hypothesis testing: given two networks with the same set of nodes, we wish to test whether the underlying Bernoulli probability matrices of the two networks are the same or not. We propose Interlacing Balance Measure (IBM) as a new two-sample testing approach. We consider the Degree-Corrected Mixed-Membership (DCMM) model for undirected networks, where we allow severe degree heterogeneity, mixed-memberships, flexible…

社会网络的混合社群归属估计AI 解读

In economics and social science, network data are regularly observed, and a thorough understanding of the network community structure facilitates the comprehension of economic patterns and activities. Consider an undirected network with n nodes and K communities. We model the network using the Degree-Corrected Mixed-Membership (DCMM) model, where for each node i=1,2,…,n, there exists a membership vector πi=(πi(1),πi(2),…,πi(K))′, where πi(k) is the weight that node i puts on…

网络社区数量的最优估计AI 解读

In network analysis, how to estimate the number of communities K is a fundamental problem. We consider a broad setting where we allow severe degree heterogeneity and a wide range of sparsity levels, and propose Stepwise Goodness of Fit (StGoF) as a new approach. This is a stepwise algorithm, where for m=1,2,…, we alternately use a community detection step and a goodness of fit (GoF) step. We adapt SCORE Jin for community detection, and propose a new GoF metric. We show that…

《统计学家的共同被引与共同署名网络》答辩回应AI 解读

We thank David Donoho for very encouraging comments. As always, his penetrating vision and deep thoughts are extremely stimulating. We are glad that he summarizes a major philosophical difference b...

网络检验中带符号多边形统计量的最优自适应性AI 解读

Given a symmetric social network, we are interested in testing whether it has only one community or multiple communities. The desired tests should (a) accommodate severe degree heterogeneity, (b) accommodate mixed memberships, (c) have a tractable null distribution and (d) adapt automatically to different levels of sparsity, and achieve the optimal phase diagram. How to find such a test is a challenging problem. We propose the Signed Polygon as a class of new tests. Fixing…

高维聚类及相关问题的相变AI 解读

Consider a two-class clustering problem where we observe $X_{i}=\ell_{i}\mu+Z_{i}$, $Z_{i}\stackrel{\mathit{i.i.d.}}{\sim}N(0,I_{p})$, $1\leq i\leq n$. The feature vector $\mu\in R^{p}$ is unknown but is presumably sparse. The class labels $\ell_{i}\in\{-1,1\}$ are also unknown and the main interest is to estimate them. We are interested in the statistical limits. In the two-dimensional phase space calibrating the rarity and strengths of useful features, we find the precise…

高维聚类中的重要特征主成分分析AI 解读

We consider a clustering problem where we observe feature vectors $X_{i}\in R^{p}$, $i=1,2,\ldots,n$, from $K$ possible classes. The class labels are unknown and the main interest is to estimate them. We are primarily interested in the modern regime of $p\gg n$, where classical clustering methods face challenges. We propose Influential Features PCA (IF-PCA) as a new clustering procedure. In IF-PCA, we select a small fraction of features with the largest Kolmogorov–Smirnov…

用于大型精度矩阵估计的偏相关筛选及其分类应用AI 解读

Given $n$ samples $X_{1},X_{2},\ldots,X_{n}$ from $N(0,\Sigma)$, we are interested in estimating the $p\times p$ precision matrix $\Omega=\Sigma^{-1}$; we assume $\Omega$ is sparse in that each row has relatively few nonzeros. We propose Partial Correlation Screening (PCS) as a new row-by-row approach. To estimate the $i$th row of $\Omega$, $1\leq i\leq p$, PCS uses a Screen step and a Clean step. In the Screen step, PCS recruits a (small) subset of indices using a…

Consider a linear model $Y=X\beta+z$, where $X=X_{n,p}$ and $z\sim N(0,I_{n})$. The vector $\beta$ is unknown but is sparse in the sense that most of its coordinates are $0$. The main interest is to separate its nonzero coordinates from the zero ones (i.e., variable selection). Motivated by examples in long-memory time series (Fan and Yao [Nonlinear Time Series: Nonparametric and Parametric Methods (2003) Springer]) and the change-point problem (Bhattacharya [In Change-Point…

稀疏高斯图模型中的最优分类AI 解读

Consider a two-class classification problem where the number of features is much larger than the sample size. The features are masked by Gaussian noise with mean zero and covariance matrix $\Sigma$, where the precision matrix $\Omega=\Sigma^{-1}$ is unknown but is presumably sparse. The useful features, also unknown, are sparse and each contributes weakly (i.e., rare and weak) to the classification decision. By obtaining a reasonably good estimate of $\Omega$, we formulate…

UPS 实现高维变量选择的最优相图AI 解读

Consider a linear model Y = Xβ + z, z ∼ N(0, In). Here, X = Xn,p, where both p and n are large, but p > n. We model the rows of X as i.i.d. samples from N(0, 1/n Ω), where Ω is a p × p correlation matrix, which is unknown to us but is presumably sparse. The vector β is also unknown but has relatively few nonzero coordinates, and we are interested in identifying these nonzeros. We propose the Univariate Penalization Screeing (UPS) for variable selection. This is a screen and…

异质且异方差混合分布的最优检测AI 解读

Summary The problem of detecting heterogeneous and heteroscedastic Gaussian mixtures is considered. The focus is on how the parameters of heterogeneity, heteroscedasticity and proportion of non-null component influence the difficulty of the problem. We establish an explicit detection boundary which separates the detectable region where the likelihood ratio test is shown to detect the presence of non-null effects reliably from the undetectable region where no method can do…

基于Student t统计量的高维数据分析方法：稳健性与精度AI 解读

Summary Student’s t-statistic is finding applications today that were never envisaged when it was introduced more than a century ago. Many of these applications rely on properties, e.g. robustness against heavy-tailed sampling distributions, that were not explicitly considered until relatively recently. We explore these features of the t-statistic in the context of its application to very high dimensional problems, including feature selection and ranking, the simultaneous…

相关噪声中检测稀疏信号的创新变换高批评检验AI 解读

Higher criticism is a method for detecting signals that are both sparse and weak. Although first proposed in cases where the noise variables are independent, higher criticism also has reasonable performance in settings where those variables are correlated. In this paper we show that, by exploiting the nature of the correlation, performance can be improved by using a modified approach which exploits the potential advantages that correlation has to offer. Indeed, it turns out…

大规模多重检验中零假设密度与非零效应比例估计的最优收敛速率AI 解读

An important estimation problem that is closely related to large-scale multiple testing is that of estimating the null density and the proportion of nonnull effects. A few estimators have been introduced in the literature; however, several important problems, including the evaluation of the minimax rate of convergence and the construction of rate-optimal estimators, remain open. In this paper, we consider optimal estimation of the null density and the proportion of nonnull…

非零正态均值的比例：通用预言等价与一致一致估计AI 解读

Summary Since James and Stein's seminal work, the problem of estimating n normal means has received plenty of enthusiasm in the statistics community. Recently, driven by the fast expansion of the field of large-scale multiple testing, there has been a resurgence of research interest in the n normal means problem. The new interest, however, is more or less concentrated on testing n normal means: to determine simultaneously which means are 0 and which are not. In this setting,…

强依赖下 higher criticism 检验的性质AI 解读

The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of…

稀疏正态混合模型的估计与置信集AI 解读

For high dimensional statistical models, researchers have begun to focus on situations which can be described as having relatively few moderately large coefficients. Such situations lead to some very subtle statistical problems. In particular, Ingster and Donoho and Jin have considered a sparse normal means testing problem, in which they described the precise demarcation or detection boundary. Meinshausen and Rice have shown that it is even possible to estimate consistently…

大规模多重比较中的零分布与非零效应比例估计AI 解读

An important issue raised by Efron in the context of large-scale multiple comparisons is that in many applications, the usual assumption that the null distribution is known is incorrect, and seemingly negligible differences in the null may result in large differences in subsequent studies. This suggests that a careful study of estimation of the null is indispensable. In this article we consider the problem of estimating a null normal distribution, and a closely related…

稀疏指数分布数据的错误发现率阈值估计的渐近极小极大性AI 解读

We apply FDR thresholding to a non-Gaussian vector whose coordinates Xi, i=1, …, n, are independent exponential with individual means μi. The vector μ=(μi) is thought to be sparse, with most coordinates 1 but a small fraction significantly larger than 1; roughly, most coordinates are simply ‘noise,’ but a small fraction contain ‘signal.’ We measure risk by per-coordinate mean-squared error in recovering log(μi), and study minimax estimation over parameter spaces defined by…

用于检测稀疏异质混合的 Higher Criticism 检验AI 解读

Higher criticism, or second-level significance testing, is a multiple-comparisons concept mentioned in passing by Tukey. It concerns a situation where there are many independent tests of significance and one is interested in rejecting the joint null hypothesis. Tukey suggested comparing the fraction of observed significances at a given α-level to the expected fraction under the joint null. In fact, he suggested standardizing the difference of the two quantities and forming a…