Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
高斯变分近似中Cholesky因子的解析自然梯度更新
Abstract Natural gradients can improve convergence in stochastic variational inference significantly but inverting the Fisher information matrix is daunting in high dimensions. Moreover, in Gaussian variational approximation, natural gradient updates of the precision matrix do not ensure positive definiteness. To tackle this issue, we derive analytic natural gradient updates of the Cholesky factor of the covariance or precision matrix and consider sparsity constraints…