Improving Sensitivity in A/B Tests: Integrating CUPED with Trimmed Mean Techniques
提高 A/B 检验灵敏度:结合 CUPED 与截尾均值
Accurate estimation of treatment effects in online A/B testing is challenging with zero-inflated and skewed metrics. Traditional tests, like Welch's t-test, often lack sensitivity with heavy-tailed data due to their reliance on means, as opposed to e.g., percentiles. The Controlled Experiments Using Pre-experiment Data (CUPED) technique improves sensitivity by reducing variance, yet that variance reduction is insufficient for highly skewed metrics. Alternatively, Yuen's…