论文检索

CAM：因果加性模型、高维排序搜索与惩罚回归AI 解读

We develop estimation for potentially high-dimensional additive structural equation models. A key component of our approach is to decouple order search among the variables from feature or edge selection in a directed acyclic graph encoding the causal structure. We show that the former can be done with nonregularized (restricted) maximum likelihood estimation while the latter can be efficiently addressed using sparse regression techniques. Thus, we substantially simplify the…

Consider a linear model $Y=X\beta+z$, where $X=X_{n,p}$ and $z\sim N(0,I_{n})$. The vector $\beta$ is unknown but is sparse in the sense that most of its coordinates are $0$. The main interest is to separate its nonzero coordinates from the zero ones (i.e., variable selection). Motivated by examples in long-memory time series (Fan and Yao [Nonlinear Time Series: Nonparametric and Parametric Methods (2003) Springer]) and the change-point problem (Bhattacharya [In Change-Point…

稀疏非凸学习问题的最优计算与统计收敛率AI 解读

-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized linear models with nonconvex regularization and sparse elliptical random design regression. For these problems, it is intractable to calculate the global solution due to the nonconvex formulation. In this paper, we propose an approximate regularization…

奇异值阈值矩阵去噪的极小极大风险AI 解读

An unknown $m$ by $n$ matrix $X_{0}$ is to be estimated from noisy measurements $Y=X_{0}+Z$, where the noise matrix $Z$ has i.i.d. Gaussian entries. A popular matrix denoising scheme solves the nuclear norm penalization problem $\operatorname{min}_{X}\|Y-X\|_{F}^{2}/2+\lambda\|X\|_{*}$, where $\|X\|_{*}$ denotes the nuclear norm (sum of singular values). This is the analog, for matrices, of $\ell_{1}$ penalization in the vector case. It has been empirically observed that if $X_{0}$…

凸约束最小二乘的新视角AI 解读

Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of the data vector on to this convex set; in other words, performing “least squares under a convex constraint.” Many problems in modern statistics and statistical signal processing theory are special cases of this general situation. Examples include the lasso and other…

贝叶斯探索性因子分析AI 解读

This paper develops and applies a Bayesian approach to Exploratory Factor Analysis that improves on ad hoc classical approaches. Our framework relies on dedicated factor models and simultaneously determines the number of factors, the allocation of each measurement to a unique factor, and the corresponding factor loadings. Classical identification criteria are applied and integrated into our Bayesian procedure to generate models that are stable and clearly interpretable. A…

模型平均、渐近风险与回归变量分组AI 解读

This paper examines the asymptotic risk of nested least-squares averaging estimators when the averaging weights are selected to minimize a penalized least-squares criterion. We find conditions under which the asymptotic risk of the averaging estimator is globally smaller than the unrestricted least-squares estimator. For the Mallows averaging estimator under homoskedastic errors, the condition takes the simple form that the regressors have been grouped into sets of four or…

通过估计量增广模拟Lasso类问题的抽样分布AI 解读

Regularized linear regression under the ℓ1 penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an ℓ1-penalized estimator is hard to determine as the estimator is defined by an optimization problem that in general can only be solved numerically and many of its components may be exactly zero. Let S be the subgradient of the ℓ1 norm of the coefficient vector β evaluated at . We find that the joint…

多个无向图的结构追寻AI 解读

Gaussian graphical models are useful to analyze and visualize conditional dependence relationships between interacting units. Motivated from network analysis under different experimental conditions, such as gene networks for disparate cancer subtypes, we model structural changes over multiple networks with possible heterogeneities. In particular, we estimate multiple precision matrices describing dependencies among interacting units through maximum penalized likelihood. Of…

利用稀疏网格实验设计快速预测确定性函数AI 解读

Random field models have been widely employed to develop a predictor of an expensive function based on observations from an experiment. The traditional framework for developing a predictor with random field models can fail due to the computational burden it requires. This problem is often seen in cases where the input of the expensive function is high dimensional. While many previous works have focused on developing an approximative predictor to resolve these issues, this…

稀疏半参数非线性模型及其色谱指纹应用AI 解读

Traditional Chinese herbal medications (TCHMs) are comprised of a multitude of compounds and the identification of their active composition is an important area of research. Chromatography provides a visual representation of a TCHM sample's composition by outputting a curve characterized by spikes corresponding to compounds in the sample. Across different experimental conditions, the location of the spikes can be shifted, preventing direct comparison of curves and forcing…

超高维纵向数据的非参数独立筛选与结构识别AI 解读

Ultra-high dimensional longitudinal data are increasingly common and the analysis is challenging both theoretically and methodologically. We offer a new automatic procedure for finding a sparse semivarying coefficient model, which is widely accepted for longitudinal data analysis. Our proposed method first reduces the number of covariates to a moderate order by employing a screening procedure, and then identifies both the varying and constant coefficients using a group SCAD…

通过切片逆回归进行一般指标模型的变量选择AI 解读

Variable selection, also known as feature selection in machine learning, plays an important role in modeling high dimensional data and is key to data-driven scientific discoveries. We consider here the problem of detecting influential variables under the general index model, in which the response is dependent of predictors through an unknown function of one or more linear combinations of them. Instead of building a predictive model of the response given combinations of…

大规模 t 检验中错误发现率控制的相变与正则化自助法AI 解读

Applying the Benjamini and Hochberg (B–H) method to multiple Student’s $t$ tests is a popular technique for gene selection in microarray data analysis. Given the nonnormality of the population, the true $p$-values of the hypothesis tests are typically unknown. Hence it is common to use the standard normal distribution $N(0,1)$, Student’s $t$ distribution $t_{n-1}$ or the bootstrap method to estimate the $p$-values. In this paper, we prove that when the population has the…

利用正则化 JIVE 处理大量弱工具变量的工具变量估计AI 解读

We consider instrumental variables regression in models where the number of available instruments may be larger than the sample size and consistent model selection in the first stage may not be possible. Such a situation may arise when there are many weak instruments. With many weak instruments, existing approaches to first-stage regularization can lead to a large bias relative to standard errors. We propose a jackknife instrumental variables estimator (JIVE) with…

通过有效平衡得分估计响应均值AI 解读

We introduce effective balancing scores for estimation of the mean response under a missing at random mechanism. Unlike conventional balancing scores, the effective balancing scores are constructed via dimension reduction free of model specification. Three types of effective balancing scores are introduced: those that carry the covariate information about the missingness, the response, or both. They lead to consistent estimation with little or no loss in efficiency. Compared…

中等高维数据中复相关系数的统计推断AI 解读

When the multiple correlation coefficient is used to measure how strongly a given variable can be linearly associated with a set of covariates, it suffers from an upward bias that cannot be ignored in the presence of a moderately high dimensional covariate. Under an independent component model, we derive an asymptotic approximation to the distribution of the squared multiple correlation coefficient that depends on a simple correction factor. We show that this approximation…

贝叶斯桥估计AI 解读

Summary We propose the Bayesian bridge estimator for regularized regression and classification. Two key mixture representations for the Bayesian bridge model are developed: a scale mixture of normal distributions with respect to an α-stable random variable; a mixture of Bartlett–Fejer kernels (or triangle densities) with respect to a two-component mixture of gamma random variables. Both lead to Markov chain Monte Carlo methods for posterior simulation, and these methods turn…

含内生变量的面板数据模型之非参数分析AI 解读

This paper considers the estimation of panel data models by first differences in the presence of endogenous variables and under an instrumental variables condition. This framework leads to the resolution of linear inverse problems solved using a Tikhonov regularization with L2 or Sobolev penalty. Rates of convergence and data driven selection of the regularization parameters are proposed. The practical implementation of our estimators is presented and some Monte Carlo…

神经影像表型与遗传标记的贝叶斯广义低秩回归AI 解读

We propose a Bayesian generalized low rank regression model (GLRR) for the analysis of both high-dimensional responses and covariates. This development is motivated by performing searches for associations between genetic variants and brain imaging phenotypes. GLRR integrates a low rank matrix to approximate the high-dimensional regression coefficient matrix of GLRR and a dynamic factor model to model the high-dimensional covariance matrix of brain imaging phenotypes. Local…

基于最小描述长度原则的分段分位数回归模型选择AI 解读

This article proposes new model-fitting techniques for quantiles of an observed data sequence, including methods for data segmentation and variable selection. The main contribution, however, is in providing a means to perform these two tasks simultaneously. This is achieved by matching the data with the best-fitting piecewise quantile regression model, where the fit is determined by a penalization derived from the minimum description length principle. The resulting…

超高维数据的交互项筛选AI 解读

In ultrahigh-dimensional data analysis, it is extremely challenging to identify important interaction effects, and a top concern in practice is computational feasibility. For a dataset with n observations and p predictors, the augmented design matrix including all linear and order-2 terms is of size n × (p2 + 3p)/2. When p is large, say more than tens of hundreds, the number of interactions is enormous and beyond the capacity of standard machines and software tools for…

鞅差相关及其在高维变量筛选中的应用AI 解读

In this article, we propose a new metric, the so-called martingale difference correlation, to measure the departure of conditional mean independence between a scalar response variable V and a vector predictor variable U. Our metric is a natural extension of distance correlation proposed by Székely, Rizzo, and Bahirov, which is used to measure the dependence between V and U. The martingale difference correlation and its empirical counterpart inherit a number of desirable…

用于超高维特征筛选的稀疏最大似然估计AI 解读

Feature selection is fundamental for modeling the high dimensional data, where the number of features can be huge and much larger than the sample size. Since the feature space is so large, many traditional procedures become numerically infeasible. It is hence essential to first remove most apparently non-influential features before any elaborative analysis. Recently, several procedures have been developed for this purpose, which include the sure-independent-screening (SIS)…