When Do Surrogate Metrics Work? A Finite-Sample Comparison Under Realistic Failure Modes
替代指标何时有效?现实失效机制下的有限样本比较
Online experiments must often be evaluated before long-term outcomes mature. Under rolling enrollment, these outcomes are observed only for early enrollees, while short-term surrogates are available for everyone. We compare seven estimators across eleven data-generating processes, spanning partial mediation, drift, outcome sparsity, and enrollment-time labeling, with up to $R = 2,000$ replications over more than 500 method-by-scenario cells. We find a sharp…