Poisson empirical Bayes estimation: When does g-modeling beat f-modeling in theory (and in practice)?

原文摘要

Empirical Bayes (EB) is a popular framework for large-scale inference that aims to find data-driven estimators to compete with the Bayesian oracle that knows the true prior. Two principled approaches to EB estimation have emerged over the years: f-modeling, which constructs an approximate Bayes rule by estimating the marginal distribution of the data, and g-modeling, which estimates the prior from data and then applies the learned Bayes rule. For the Poisson model, the prototypical examples are the celebrated Robbins estimator and the nonparametric MLE (NPMLE), respectively. It has long been recognized in practice that the Robbins estimator, while being conceptually appealing and computationally simple, lacks robustness and can be easily derailed by “outliers” (data points that were rarely observed before), unlike the NPMLE which provides more stable and interpretable fit thanks to its Bayes form. On the other hand, not only do the existing theories shed little light on this phenomenon, but they all point to the opposite, as both methods have recently been shown optimal in terms of the regret (excess over the Bayes risk) for compactly supported and subexponential priors with exact logarithmic factors (Brown, Greenshtein and Ritov (2013) J. Amer. Statist. Assoc. 108 741–749; Polyanskiy and Wu (2021) arXiv:2008.08244). In this paper we provide a theoretical justification for the superiority of g-modeling over f-modeling for heavy-tailed data by considering priors with bounded pth moment, a class previously studied for the Gaussian model (Jiang and Zhang (2009) Ann. Statist. 37 1647–1684). For the Poisson model with sample size n, assuming p>1 (for otherwise triviality arises), we show that with mild regularization, any g-modeling method that is Hellinger rate-optimal in density estimation achieves a total regret Θ˜(n 32p+1), which is minimax optimal within logarithmic factors; in particular, the special case of NPMLE succeeds without regularization. In constrast, there exists an f-modeling estimator whose density estimation rate is optimal but whose EB regret is suboptimal by a polynomial factor. These results show that the proper Bayes form provides a “general recipe of success” for optimal EB estimation that applies to all g-modeling (but not f-modeling) methods. As by-products of our analysis, we also obtain: (a) the minimax Hellinger rate of estimating Poisson mixture over the moment class, (b) the characterization of the regret suboptimality of the Robbins estimator, (c) an extension to the compound setting.

摘要来源：Crossref