Peter J. Bickel, Derek Bean, Aiyou Chen and Purnamrita Sarkar's contribution to the Discussion of ‘Vintage Factor Analysis with Varimax Performs Statistical Inference’ by Rohe & Zeng

原文摘要

We compliment the authors for their unification of old and new latent variable models. Their motivation seems to have come from Thurstone’s Factor Analysis and the wish to explain the somewhat mysterious success of Kaiser’s Varimax rule in that context. However, their major applications and notation stem from overlapping network community detection and Latent Dirichlet Allocation (LDA) (Blei et al., 2003). Inference for these models can be unified in another way via a provably consistent algorithmic framework (Mao et al., 2018) because of the inherent similarities in the geometrical structure of eigenvectors of appropriate matrices. It would be interesting to see how the identifiability conditions tied to these models, which typically involve the existence of pure nodes or anchor words relate to those needed for Varimax. We elaborate on a connection to Independent Component Analysis (ICA) (Hyvärinen, 2013; Hyvärinen et al., 2001), which they have noted, and make explicit a connection between Varimax and the FastICA algorithm of Hyvarinen (1999a). The factor analysis/ICA model we consider has n independent observations with the following structure: where A and E are d×1⁠, Z is a K×1 random vector independent of E, and B is d×K⁠, full rank and nonrandom. The elements of Z are independent with no Gaussian component, and have mean zero and unit variance. Also, E∼N(0,Σ)⁠, with unknown covariance matrix Σ. This noisy ICA model is a special case of the model of this paper. When d=K⁠, the goal is to estimate a nonsingular C such that CA=PZ+CE⁠, where P is some permutation matrix. If E=0 and A is prewhitened, C (and B) are orthogonal yielding factors with Z as the vector of loadings. By a theorem of Comon (1994), B is identifiable up to a permutation as is Σ. FastICA (Hyvarinen, 1999a, 1999b), which estimates C by maximizing the sum of the squared empirical kurtoses of the coordinates of CA after prewhitening, yields consistent estimates if all the Zj have nonzero kurtoses and E=0 or Σ is known. The kurtosis assumption can be dropped (Chen & Bickel, 2005) using a different fast algorithm. The authors’ main Theorem 6.1 requires ‘leptokurtosis’. Is that necessary? Another latent variable model of interest is NonGaussian Components Analysis (Blanchard et al., 2006), where A is viewed as the sum of an unknown nonGaussian signal and independent Gaussian noise with the aim of estimating the linear space in which the signal lies. Is there a relation to the model of this paper?

摘要来源：OpenAlex