Yunxiao Chen and Gongjun Xu's contribution to the Discussion of ‘Vintage Factor Analysis with Varimax Performs Statistical Inference’ by Rohe & Zeng

原文摘要

We congratulate Rohe and Zeng for this insightful paper that elegantly connects psychometric methods and statistical and machine learning applications. We would like to mention several lines of related research. First, the problem is closely related to the latent variable selection problems (e.g., Y. Chen et al., 2015; Xu & Shang, 2018), where regularised estimation procedures are proposed to learn sparse loading structures. In fact, the vsp procedure can be viewed as the limiting case of a regularised estimation procedure, in the sense that U^ from Algorithm vsp is the limit of when λ goes to zero if the solution path is smooth, where U and V satisfy the same constraints as in singular value decomposition, while D is allowed to be non-diagonal (F. Chen & Rohe, 2020). Note that the regularisation is used to learn the sparse loading structure rather than to avoid over-fitting, and thus, it does not require the tuning parameter to depend on the noise level. We agree that rotation is more convenient under many models, but the regularised estimation approach might be more general for some more complex latent variable models. Second, various rotation methods have been proposed in the psychometric literature to find simple and scientifically meaningful factor loading structures. For example, consider the L1 criterion that minimises the objective function This criterion is closely related to L1 regularisation and ensures statistical consistency under suitable conditions (Jennrich, 2004, 2006; Liu et al., 2022). We run a small simulation study to compare the Varimax and L1 rotations, where data are generated from the current factor model. Two settings are used to generate Z—one sparse setting where P([Z]ij=0)=0.5 and one dense setting where [Z]ij follows a heavy-tail distribution. The L1 rotation method only replaces v(R,U^) in step 3 of the vsp algorithm with c(R,U^)⁠. The mean squared errors for the estimation of Z are given in Figure 1, where the L1 rotation performs better under the sparse setting while the vsp outperforms under the dense setting. Box plots of mean squared errors ‖Z^−ZPn‖F2/(nk) from 100 simulations. The four box plots correspond to the combinations of two settings (⁠k=5,n=d=100,200⁠) and two rotation methods (Varimax and L1⁠). In the simulations, we generate A=ZYT+W⁠, where [A]ij and [W]ij are independent standard normal variables. Under Setting 1, [Z]ij=2[C]ij[S]ij⁠, where [C]ij are independent standard normal random variables and [S]ij are independent Bernoulli random variables with success probability 0.5. Under Setting 2, [Z]ij=[T]ij/5/3⁠, where [T]ij follows a t distribution with five degrees of freedom. R code for the simulation can be found on https://stats.lse.ac.uk/cheny185/L1˙rotation.R. Finally, another interesting extension of the current work is to non-linear factor models that assume E([A]ij|Z,B,Y)=f([ZBYT]ij)⁠, for some known smooth and strictly monotone non-linear function f (e.g., logistic function for binary data). Due to the non-linear transformation, the current vsp procedure does not directly apply. One solution is to first apply the universal singular value thresholding procedure (Chatterjee, 2015) to A to estimate (f([ZBYT]ij))n×d⁠, which yields an estimate of ZBYT through element-wise f−1 transformations; see Zhang et al. (2020) for more details and the related consistency theory. Then, one can learn Z by steps 2 and 3 of Algorithm vsp. G.X. is supported by National Science Foundation grants SES-1846747 and SES2150601. The data supporting the findings of this study are available within the article.

摘要来源：OpenAlex