Kaizheng Wang's contribution to the Discussion of ‘Vintage Factor Analysis with Varimax Performs Statistical Inference’ by Rohe & Zeng

原文摘要

I congratulate Drs. Rohe and Zeng for their elegant paper that advances our understanding of the classical Varimax algorithm. Varimax was originally proposed by Kaiser (1958) as an analytic criterion for determining interpretable factors. Despite its extensive applications and tremendous success, theoretical understanding is still lacking. Statisticians are further perplexed by negative results showing the nonidentifiability of Gaussian factor models. Toward resolving the dilemma, Rohe and Zeng proved that Varimax performed on the spectral embedding of data indeed recovers meaningful structures under a wide class of semi-parametric factor models. Below I briefly summarize their key insights and discuss new avenues of research. The authors assumed that the observed data A∈Rn×d satisfies E(A|Z,Y)=ZBY⊤⁠, where Z∈Rn×K⁠, Y∈Rd×K are random latent factor matrices and B∈RK×K a deterministic matrix. The ith row zi of Z consists of latent factors of the ith sample, e.g., community membership of an individual, topics of a document, etc. The authors showed that if elements of zi are independent and leptokurtic, then Varimax consistently estimates them. Interestingly, leptokurticity can be guaranteed by sparsity (Theorem 4.1), which is ubiquitous in high-dimensional models. The paper opens up exciting possibilities for further study. Recall that Varimax searches for a K×K orthonormal matrix that maximizes the sum of kurtoses of the K transformed factors. However, kurtosis maximization fails to recover factors if the leptokurticity assumption is violated. As a toy example, consider K=d=2⁠, A=Z and let {zi}i=1n be i.i.d. from the degenerate Gaussian mixture 12N(e1,e2e2⊤)+12N(−e1,e2e2⊤)⁠. The two canonical coordinates correspond to independent factors and the first one is platykurtic. Elementary calculation shows that as n→∞⁠, Varimax rotates the data by 45 degrees and returns noninterpretable coordinates. So, how should one choose the objective function in general? Hyvarinen (1997) showed advantages of several objectives over the kurtosis. More research is needed for modern statistical models. Another question is whether one should compute an orthonormal matrix in one shot or K orthogonal directions one by one. The latter may have computational advantage while the former is more stable numerically. Moreover, they generally yield different solutions even as n→∞⁠. Their statistical properties are worth investigating. Last but not least, the nonconvex objective functions call for studies of efficient algorithms with convergence guarantees.

摘要来源：OpenAlex 索引摘要（稿件版本未注明）