论文检索

用于词嵌入的词级最大均值差异正则化AI 解读

The technique of word embedding is widely used in natural language processing (NLP) to represent words as numerical vectors in textual datasets. However, the estimation of word embedding may suffer from severe overfitting due to the huge variety of words. To address the issue, this article proposes a novel regularization framework that recognizes and accounts for the “word-level distribution discrepancy”–a common phenomenon in a range of NLP tasks where word distributions…

无需估计秩的低秩模型推断AI 解读

This paper studies the inference about linear functionals of high-dimensional low-rank matrices. While most existing inference methods would require consistent estimation of the true rank, our procedure is robust to rank misspecification, making it a promising approach in applications where rank estimation can be unreliable. We estimate the low-rank spaces using pre-specified weighting matrices, known as diversified projections. A novel statistical insight is that, unlike…

高维投资组合优化中的样本外夏普比率估计AI 解读

Portfolio optimization aims at constructing a realistic portfolio with significant out-of-sample performance, which is typically measured by the out-of-sample Sharpe ratio. However, due to in-sample optimism, it is inappropriate to use the in-sample estimated covariance to evaluate the out-of-sample Sharpe, especially in the high dimensional settings. In this paper, we propose a novel method to estimate the out-of-sample Sharpe ratio using only in-sample data, based on…

高维向量自回归的数据驱动调参AI 解读

Lasso-type estimators are routinely used to estimate high-dimensional time series models. The theoretical guarantees established for these estimators typically require the penalty level to be chosen in a suitable fashion often depending on unknown population quantities. Furthermore, the resulting estimates and the number of variables retained in the model depend crucially on the chosen penalty level. However, there is currently no theoretically founded guidance for this…

高维空间面板网络的统一推断AI 解读

We propose employing a high-dimensional generalized method of moments (GMM) estimator, regularized for dimension reduction and subsequently debiased to correct for shrinkage bias (referred to as a debiased-regularized estimator), for inference on large-scale spatial panel networks. In particular, the network structure, which incorporates a flexible sparse deviation that can be regarded either as a latent component or as a misspecification of a predetermined adjacency matrix,…

高维函数时间序列的因子建模AI 解读

Many economic and scientific problems involve the analysis of high-dimensional functional time series, where the number of functional variables p diverges as the number of serially dependent observations n increases. In this paper, we present a novel functional factor model for high-dimensional functional time series that maintains and makes use of the functional and dynamic structure to achieve great dimension reduction and find the latent factor structure. To estimate the…

具有不可观测分组异质性的面板自回归模型统一推断AI 解读

This paper considers unified estimation and inference in panel autoregressive (PAR) models. The PAR coefficients are assumed to contain a latent group structure that allows the degree of persistence for each time series to be heterogeneous and unknown. We propose a novel penalized weighted least squares approach to simultaneously identify the unknown group membership and consistently estimate the PAR coefficients, regardless of whether the underlying PAR process is…

高维控制变量下条件平均处理效应的双重稳健推断AI 解读

Plausible identification of conditional average treatment effects (CATEs) can rely on controlling for a large number of variables to account for confounding factors. In these high-dimensional settings, estimation of the CATE requires estimating first-stage models whose consistency relies on correctly specifying their parametric forms. While doubly-robust estimators of the CATE exist, inference procedures based on the second-stage CATE estimator are not doubly-robust. Using…

高维广义矩阵因子模型的估计与推断AI 解读

This article introduces a nonlinear generalized matrix factor model, moving beyond the linear-Gaussian framework to accommodate a broader class of response models typically handled via logit, probit, Poisson, or Tobit structures. We introduce a novel Lagrange multiplier method carefully tailored to ensure that the penalized likelihood function is locally concave around the true factor and loading parameters. This leads to central limit theorems of the estimated factors and…

非平稳与周期时间序列的函数型半参数建模AI 解读

Inspired by a real data example illustrating the periodicity in hog price data, this study aims to analyze time series that exhibit an unknown period alongside complex covariate effects. To address these complexities and effectively handle the data structures, we incorporate the partial functional varying-coefficient single-index model into the classical time series decomposition model. We propose a two-stage estimation procedure designed to accurately estimate the unknown…

含交互固定效应的高维离散选择模型及其因果推断应用AI 解读

ABSTRACT We propose a two‐step procedure to estimate a high dimensional discrete choice panel with interactive fixed effects where the initial and final estimators are obtained via a nuclear‐norm regularized (NNR) maximum likelihood estimation and post‐NNR iterated estimation, respectively. We apply the method to make counterfactual predictions of choice probabilities. Simulations demonstrate nice finite sample performance in estimation and tests. An illustrative application…

希尔伯特流形变量的高维Hilbert–Schmidt线性回归AI 解读

In this paper, we propose a novel high-dimensional linear regression technique for variables taking values in Hilbert manifolds. Our approach offers a flexible framework where the response and covariates originate from distinct spaces and are interconnected by Hilbert–Schmidt operators. Our methodology is designed for situations where the number of the covariates grows exponentially fast as the sample size increases and some of the covariates together with the response take…

核岭回归的几何分析及其应用AI 解读

We obtain upper bounds for the estimation error of Kernel Ridge Regression (KRR) for all nonnegative regularization parameters, offering a geometric perspective on various phenomena in KRR. As applications: 1. We address the Multiple Descents problem, unifying the proofs of (In Proceedings of Thirty Third Conference on Learning Theory (2020) 2683–2711 PMLR) and (Ann. Statist. 49 (2021) 1029–1054) for polynomial kernels in nonasymptotic regime and we establish Multiple…

完备性是否必要？未识别线性模型中的估计AI 解读

Modern data analysis depends increasingly on estimating models via flexible high-dimensional or nonparametric machine learning methods, where the identification of structural parameters is often challenging and untestable. In linear settings, this identification hinges on the completeness condition, which requires the nonsingularity of a high-dimensional matrix or operator and may fail for finite samples or even at the population level. Regularized estimators provide a…

可能包含大量协变量的断点回归设计AI 解读

This article examines high-dimensional covariates in regression discontinuity design (RDD) analysis. We introduce estimation and inference methods for the RDD models that incorporate covariate selection while maintaining stability across various numbers of covariates. The proposed methods combine a localization approach using kernel weights with $\ell _{1}$ -penalization to handle high-dimensional covariates. We provide both theoretical and numerical evidence demonstrating…

Abstract For data segmentation in high-dimensional linear regression settings, the regression parameters are often assumed to be exactly sparse segment-wise, which enables many existing methods to estimate the parameters locally via ℓ1-regularized maximum-likelihood-type estimation and then contrast them for change point detection. Contrary to this common practice, we show that the exact sparsity of neither regression parameters nor their differences, a.k.a. differential…

重尾高频金融观测下的因子与特异性 VAR 波动率矩阵模型AI 解读

This paper introduces a novel process for both factor and idiosyncratic volatility matrices whose eigenvalues follow the vector auto-regressive (VAR) model. We call it the factor and idiosyncratic VAR (FIVAR) model. The FIVAR model accounts for the dynamics of the factor and idiosyncratic volatilities and includes many parameters. In addition, many empirical studies have shown that high-frequency stock returns and volatilities often exhibit heavy tails. To handle these two…

使高维分布稳健投资组合问题可行AI 解读

Robust estimation for modern portfolio selection on a large set of assets becomes more important due to the large deviation of empirical inference on big data. We propose a distributionally robust methodology for high-dimensional mean–variance portfolio problems, aiming to select an optimal conservative portfolio allocation by considering distributional uncertainty. With the help of factor structure, we extend the distributionally robust mean–variance problem investigated by…

高维线性回归中低维参数推断的正交投影方法AI 解读

We consider the estimation and statistical inference for low dimensional parameters for a regression model with covariates whose dimension increases with sample size. We suggest a computationally simple one stage orthogonal projection approach to estimate the low dimensional parameters under strict or approximate sparsity conditions. The orthogonal projection approach is simple to implement and the inference for the low dimensional parameters is straightforward to derive…

加性模型下异质处理效应估计量的模型平均AI 解读

We consider spline-based additive models for estimation of conditional treatment effects. To handle the uncertainty due to variable selection, we propose a method of model averaging with weights obtained by minimizing a J -fold cross-validation criterion, in which a nearest neighbor matching is used to approximate the unobserved potential outcomes. We show that the proposed method is asymptotically optimal in the sense of achieving the lowest possible squared loss in some…

更稳健的多变量孟德尔随机化方法AI 解读

Summary Multivariable Mendelian randomization uses genetic variants as instrumental variables to infer the direct effects of multiple exposures on an outcome. However, unlike univariable Mendelian randomization, multivariable Mendelian randomization often faces greater challenges with many weak instruments, which can lead to bias not necessarily toward zero and inflation of Type-I errors. In this work, we introduce a new asymptotic regime that allows exposures to have…

稳健的高维时变系数估计AI 解读

In this article, we develop a novel high-dimensional coefficient estimation procedure based on high-frequency data. Unlike usual high-dimensional regression procedures such as LASSO, we additionally handle the heavy-tailedness of high-frequency observations as well as time variations of coefficient processes. Specifically, we employ the Huber loss and a truncation scheme to handle heavy-tailed observations, while $\ell _{1}$ -regularization is adopted to overcome the curse…

RLHF 对齐的算法偏差：偏好坍缩与匹配正则化AI 解读

Accurately aligning large language models (LLMs) with human preferences is crucial for informing fair, economically sound, and statistically efficient decision-making processes. However, we argue that the predominant approach for aligning LLMs with human preferences through a reward model—reinforcement learning from human feedback (RLHF)—suffers from an inherent algorithmic bias due to its Kullback–Leibler-based regularization in optimization. In extreme cases, this bias…

具有预言性质与几何收敛性的通信高效分布式稀疏学习AI 解读

This article introduces two highly efficient distributed non-convex sparse learning algorithms. Our approach accommodates non-convexity in both the loss function and penalty, acknowledging the potential non-uniqueness of local minimizers due to the inherent non-convexity. The development of an algorithm that ensures convergence to a locally minimal solution with desired statistical properties becomes imperative in this context. To overcome this challenge, we propose a…