论文检索

控制错误发现率与功效的贝叶斯 A/B 测试设计AI 解读

Businesses frequently run online controlled experiments (i.e., A/B tests) to learn about the effect of an intervention on multiple business metrics. To account for multiple hypothesis testing, multiple metrics are commonly aggregated into a single composite measure, losing valuable information, or strict family-wise error rate adjustments are imposed, leading to reduced power. In this paper, we propose an economical framework to design Bayesian A/B tests while controlling…

贝叶斯后验分析设计的低成本方法AI 解读

To design Bayesian studies, criteria for the operating characteristics of posterior analyses – such as power and the type I error rate – are often assessed by estimating sampling distributions of posterior probabilities via simulation. In this paper, we propose an economical method to determine optimal sample sizes and decision criteria for such studies. Using our theoretical results that model posterior probabilities as a function of the sample size, we assess operating…