Neil Dey, Ryan Martin, and Jonathan P. Williams’ contribution to the Discussion of ‘Safe Testing’ by Grünwald, de Heide, and Koolen

原文摘要

Congratulations to Grünwald, de Heide, and Koolen (GHK) for their contribution to the exciting and rapidly growing literature on safe/anytime-valid inference. GHK’s proposal focused on problems in which a statistical model is available that determines the quantity of interest—the model parameter—and provides a likelihood function that drives the e-value construction. When data are observed and analysed sequentially, however, the data analyst typically cannot say with any certainty what statistical model might be appropriate. Our comments below focus on what can be done along the lines of safe/anytime-valid inference without a correctly specified statistical model. Two recent developments deserve mention. First, Park, Balakrishnan, and Wasserman, in their recent arXiv paper (https://arxiv.org/abs/2307.04034) develop a version of the universal inference framework (Wasserman et al., 2020) that accommodates model misspecification but is not designed for the online setting. Second, the nonparametric approach in Waudby-Smith and Ramdas (2024) offers anytime-valid inference, but focuses exclusively on the mean of the data-generating process. In our recent arXiv paper (https://arxiv.org/abs/2402.00202), we take a different approach, starting from the basic question: What is the inferential target? From classical M-estimation to modern machine learning, the inferential target is often the solution to a suitable optimization problem. These solutions can be cast as risk minimizers: Given a loss function ℓθ(z)⁠, the inferential target is θ⋆=argminθR(θ)⁠, with R(θ)=∫ℓθ(z)P(dz) the risk. We generalize the universal inference framework by replacing the statistical model’s negative log-likelihood with a multiple of the empirical risk, Rn(θ)=n−1∑i=1nℓθ(Zi)⁠, and prove that it offers safe, anytime-valid inference on risk minimizers. Circling back to GHK’s focus on growth rate optimality, we show that our generalized universal e-process for testing H0:θ⋆∈Θ0 vs. H1:θ⋆∈Θ1⁠, has (asymptotic) log-growth rate The above claim is analogous to Theorem 2 in the recent arXiv report by Dixit and Martin (https://arxiv.org/abs/2309.13441). Our conjecture is that (1) gives the ‘optimal’ asymptotic first-order growth rate achievable by an e-process relative to the given learning problem (P,ℓ)⁠. If our conjecture is true, then given the similarities between GHK’s growth rate and (1), one could approach this more general problem similarly to GHK, by defining an e-process based on a growth rate optimality criterion, and we expect its logarithm would be proportional to an empirical risk difference. Further, the corresponding ‘growth rate optimal’ e-process would have to be similar to what we proposed in our aforementioned paper.

摘要来源：OpenAlex 索引摘要（稿件版本未注明）