Evaluating A/B Testing Methodologies via Sample Splitting: Theory and Practice
通过样本分割评估 A/B 测试方法:理论与实践
We develop a theoretical framework for sample splitting in A/B testing environments, where data for each test are partitioned into two splits to measure methodological performance when the true impacts of tests are unobserved. We show that sample-split estimators are generally biased for full-sample performance but consistently estimate sample-split analogues of it. We derive their asymptotic distributions, construct valid confidence intervals, and characterize the…