The Noise Is the Signal: Correlated Sampling Error Is Rank-Informative for Proxy Metric Selection
噪声即信号:相关抽样误差为代理指标选择提供排序信息
North-star metrics such as customer lifetime value are often too slow and noisy to decide a short A/B test. Teams therefore rely on a proxy metric, commonly chosen by how closely its effects tracked the north star's across past experiments. Validating that choice, or any method for making it, is hard: the only benchmark is the noisy north star, and the number of available past experiments is limited. In addition, proxy and north-star effects are estimated on the same…