Yang Liu's contribution to the Discussion of ‘Vintage Factor Analysis with Varimax Performs Statistical Inference’ by Rohe & Zeng

原文摘要

I congratulate Rohe and Zeng for the fascinating piece of work. They discovered that sparse principle components analysis (PCA) with Varimax rotation (vsp) identifies and consistently recovers latent factors in a broad class of semi-parametric factor models. I have a number of comments to share regarding identification assumptions, asymptotic results, and interpretation of factors. Identification Assumptions. The identification of the low-rank matrix M=ZBY⊤ (when treated as fixed) has been well understood in the literature of matrix completion (e.g., Chatterjee, 2015; Fan et al., 2019). The main contribution of the present paper is to establish that, not only M, but also the factorization of M can be uniquely identified when the factors are random and sparse/leptokurtic. The connection between sparseness and leptokurticity is an intriguing standalone result. However, the sparsity requirement in Theorem 4.1 is still restrictive in some applications when k is small: For example, a bifactor structure (Holzinger & Swineford, 1937) wherein each row of Z (or Y) has two non-zero entries does not satisfy the condition if k<12⁠. I am wondering if results similar to Theorem 5.1 (i.e., uniqueness of rotation) can be established under more general sparseness conditions. Asymptotic Result. Seeing the close resemblance between vsp and matrix completion, it is natural to ask how different their asymptotic rates of convergence are. In matrix completion, the low-rank matrix M is considered fixed. I conjecture that an error bound for an identifiable Z may be obtained by combining an error bound for the low-rank matrix M; an error bound for the singular vectors U (up to a rotation matrix); Proposition 5.2 that relates U to Z. Such a result is not yet readily comparable to Theorem 6.1 as M is random in the present paper; however, it may be possible to conclude that the bound holds with a large probability given the distributional assumptions for Z and Y. Considering that vsp is computationally much cheaper, I do not expect that it outperforms matrix completion in accuracy. Interpretation of Factors. My following comments are based on a more ‘vintage’ specification of factor analysis, in which Z is random and unstructured (i.e., scores) and Y~=YB⊤ is fixed and sparse (i.e., loadings). First, factor analysis is not just about finding a low-dimensional approximation to the data—it also implies that the low-dimensional factors fully account for the dependencies in data (Fabrigar et al., 1999). Second, Thurstone’s reference to ‘scientifically meaningful category’ cannot be simply translated to ‘statistically identifiable factors’. It is sometimes meaningful to have two different explanations, corresponding to two differently rotated solutions, on the same identifiable, low-dimensional structure. Here is a toy example deduced from the well-known Schmid-Leiman transform (Schmid & Leiman, 1957): in which $Z_1$ and $Z_2$ contain independent and identically distributed normal variates. It is obvious that Z1Y~1⊤=dZ2Y~2⊤ because Y~1Φ1Y~1⊤=Y~2Φ2Y~2⊤⁠. Both Y~1 and Y~2 have meaningful interpretations in psychological theory: The former represents correlated traits indicated by distinct observable responses, while the latter contains one overall and two specific traits that are mutually orthogonal.

摘要来源：OpenAlex