Yinqiu He, Yuqi Gu and Zhilian Ying's contribution to the Discussion of ‘Vintage Factor Analysis with Varimax Performs Statistical Inference’ by Rohe & Zeng

原文摘要

We congratulate the authors on an impressive paper that demystifies the popular Varimax rotation. They showed that the Varimax rotation can identify true axes of latent factors under the leptokurtic condition in a low-rank semiparametric factor model Different from (1), the exploratory Item Factor Analysis (IFA), widely used in social and behavioural sciences, adopts a nonlinear transformation of the low-rank structure: where the binary data matrix R contains N individuals’ responses to J items, Θ represents N individuals’ K latent factors, Λ is the loading matrix, and F is a prespecified monotone increasing function. The relationship between the two models can be seen by letting F(x)=x⁠, B=IK⁠, Z=Θ⁠, and Y=Λ⁠. However, in IFA, F(⋅) is usually taken from a distribution function, e.g., FNormal(x)=∫−∞xe−t2/2/(2π)dt and FLogistic(x)=ex/(1+ex)⁠; see Reckase (2009). For such a model, it is of interest to develop a similar Vintage Factor Analysis with Varimax. For (2), Zhang et al. (2020) proposed a two-step approach: (1) use SVD to obtain a low-rank approximation to the data matrix, denoted as R^⁠; (2) use F−1(R^) as an estimate of ΘΛ⊤ and apply SVD to obtain Θ^ and Λ^⁠. We add a third step that incorporates the Varimax rotation: (3) apply Varimax rotation to the initial estimate Θ^⁠. Our preliminary studies show some promising signs. Specifically, we conduct simulations under four settings of Θ⁠, given in Figures 1–4, respectively. In each setting, we consider three F(⋅) in (2): FNormal(x)⁠, FLogistic(x)⁠, and FCauchy(x)=π−1arctan(x)+1/2⁠. For d∈{Normal,Logistic,Cauchy}⁠, we generate a binary data matrix Rd with a population mean matrix Fd(ΘΛ⊤)⁠. Setting (I): N=J=500,K=2⁠. True factors Θ=(θik) are binary and give orthogonal columns: (θi1,θi2) are i.i.d. Multinomial(0.5,0.5)⁠. Data are not centred, as factors are orthogonal. The loading matrix Λ has i.i.d. entries following U(−2,2)⁠. Point i in red circle if θi1=0⁠, and blue triangle otherwise. The first figure in the first row gives the scatterplot of true factors Θ⁠, and the other figures plot estimated factors that are obtained following the subtitles above themselves. Setting (II): N=J=500,K=2⁠. True factors Θ=(θik) are binary and independent: for k∈{1,2}⁠, θik=norm(ζik)⁠, where ζik are i.i.d. Bernoulli(1/7)⁠, and norm(ζik) represents normalizing ζik by its population mean and standard deviation. Kurtosis(θik)>3 for k∈{1,2}⁠. Data are centred, as true factors are mean zero. The loading matrix Λ has i.i.d. entries following U(−2,2)⁠. Let ζi=(ζi1,ζi2)⁠. For i=1,…,N⁠, point i is in red circle if ζi=(0,0)⁠, in blue triangle if ζi=(0,1)⁠, in yellow square if ζi=(1,0)⁠, and in green diamond if ζi=(1,1)⁠. Setting (III): N=J=500,K=2⁠. True factor Θ=(θik) is nonbinary and give orthogonal columns: for k∈{1,2}⁠, θik=ϱik×mik⁠, where ϱik are i.i.d. Poisson(1)⁠, (mi1,mi2)∼Multinomial(0.5,0.5)⁠, and Kurtosis(θik)>3⁠. Data are not centred, as factors are orthogonal. The loading matrix Λ has i.i.d. entries following U(−2,2)⁠. Point i is in red circle if θi1=0⁠, and blue triangle otherwise. Setting (IV): N=J=500,K=2⁠. True factors Θ=(θik) are independent and include nonbinary: θi1∼norm(ζi1)⁠, where ζi1 are i.i.d. Bernoulli(1/7)⁠, and θi2∼ρi2−1⁠, where ρi2 are i.i.d. Poisson(1). Data are centred, as true factors have zero mean. The loading matrix Λ has i.i.d. entries following U(−2,2)⁠. Point i is in red circle if ζi1=0⁠, and blue triangle otherwise. (i) Population (Columns 2–4 in Figures 1–4). We apply the two-step algorithm in Zhang et al. (2020) to Fd(ΘΛ⊤) and obtain initial estimated factors Θ^P,Initial(d)⁠. We then apply Varimax to Θ^P,Initial(d) and obtain rotated factors Θ^P,Rotate(d)⁠. For d∈{Logistic,Cauchy}⁠, rotated factors Θ^P,Rotate(d) are close to true Θ⁠, whereas Θ^P,Initial(d) with d∈{Normal} can substantially deviate from Θ⁠. The results show that latent axes might be statistically identified given a nonlinear F in (2). However, the performance varies with the choice of F: FLogistic and FCauchy (heavy tail) outperform FNormal (light tail). (ii) Sample (Columns 5–7 in Figures 1–4). Similarly, we first apply the two-step algorithm to binary matrix Rd to obtain Θ^S,Initial(d)⁠, and then Θ^S,Rotate(d) via Varimax rotation. In Figures 1–2, binary Θ renders finite number of clusters, and the estimated factors can recover the clusters. This could be because in this case, (2) can be alternatively viewed as a Latent Class Model (Goodman, 1974), and Fd(ΘΛ⊤) always exhibits an exact low-rank structure even with a nonlinear Fd⁠. This observation aligns with the authors’ findings. However, in Figures 2–4, the sample estimate Θ^S,Rotate(d) can deviate from true Θ more significantly than its population counterpart Θ^P,Rotate(d) does. This may be due to the special signal-to-noise structure of a binary variable, i.e., its variance can always be computed from its mean, so the sampling noise can have a significant impact on the recovery accuracy. Nevertheless, comparing Θ^S,Initial(d) and Θ^S,Rotate(d)⁠, Varimax can indeed find one rotation aligned with the axes of the true latent factors. This suggests that similar Vintage Factor Analysis with varimax may also apply in the IFA model (2) with a nonlinear F and Θ that satisfy leptokurtic conditions. The following contributions were received in writing after the meeting.

摘要来源：OpenAlex