Philip B. Stark’s contribution to the Discussion of ‘Estimating means of bounded random variables by betting’ by Waudby-Smith and Ramdas

原文摘要

This paper is a gem, and I predict it will have lasting impact on theoretical and applied statistics. Drawing on intellectual heritage spanning more than 80 years, illustrating and leveraging deep connections among results in probability, finance, information theory, computer science, and statistics, Waudby-Smith and Ramdas present mature, intuitive, flexible, computable, and powerful methods for two fundamental (indeed, canonical) nonparametric inference problems: conservative confidence bounds for the expected value of a bounded random variable from IID observations and conservative confidence bounds for the mean of a finite, bounded population from a simple random sample. Sequentially valid methods have been proposed for those problems and the related problem of conservative tests for nonnegative or bounded means [including some based on Ville’s (1939) inequality, explicitly or through Wald’s (1945) sequential probability ratio test: Howard et al., 2021; Kaplan, 1987, 2012; Stark, 2009, 2020, 2023; Stark & Teague, 2014], but their performance varies and their derivations provide little insight into how to sharpen the methods. In contrast, the betting martingale framework leads naturally to techniques that approximately optimise power, are computationally tractable, perform well ‘out of the box’ for a broad range of bounded populations and distributions, and can incorporate prior information without compromising coverage. Moreover, the sequential validity makes them ‘safe’ against common practices that invalidate other approaches (optional stopping and optional continuation) (Grünwald et al., 2023; Ramdas et al., 2021; Shafer, 2021). It would be interesting to compare these new methods to the approach of Orabona and Jun (2021) based on regret bounds for a universal portfolio and to the conjectured bound of Gaffke (2005) and Learned-Miller and Thomas (2019) for fixed sample size, even though the latter bound has not been proved to be valid in general. Accommodating Bernoulli sampling is trivial (see, e.g. Ottoboni et al., 2020; Stark, 2023), but tied to the question of derandomisation, especially for fixed sample size. [Kaplan (1987) does this in a crude way using the union bound; Waudby-Smith and Ramdas give a number of sharper approaches, and work in progress makes further improvements; Ramdas & Manole, 2023.] Accommodating stratified sampling using union–of–intersection tests is straightforward (Spertus et al., 2023; Spertus & Stark, 2022; Stark, 2023) because independent E-values can be combined by multiplication or averaging (Vovk & Wang, 2021); alternatively, the E-values can be converted to P-values and then combined. Optimising union–of–intersection tests based on test martingales yields an interesting generalisation of the multi-arm bandit (MAB) problem, a ‘gang of bandits’, where N MABs are probed in parallel: at each stage, the statistician chooses an arm, which pulls the same arm of all N MABs, as if they were ‘ganged’ like switches. The statistician has a fortune for each MAB and bets separately on each, but seeks to maximise their minimum fortune across the bandits (Stark, 2023) by choosing the sequence of arms and bets. Improved methods for stratified inference have immediate applications ranging from election audits (Ottoboni et al., 2020; Spertus & Stark, 2022; Stark, 2008, 2009, 2020, 2023; Stark & Teague, 2014; Waudby-Smith et al., 2021) and financial audits to randomised experiments with blocking. In closing, I highly recommend appendix F for a brief history of connections among gambling, portfolio theory, martingales, concentration inequalities, and hypothesis testing.

摘要来源：OpenAlex 索引摘要（稿件版本未注明）