Identifying Model Quality Effects on User Engagement: A Within-Version Causal Estimator with Synthetic Data Validation
模型质量对用户参与度的影响:版本内因果估计与合成数据验证
Every team building Large Language Models (LLMs) faces a core challenge: offline benchmarks show performance gains and user engagement rises post deployment, but isolating cause from effect remains difficult. Simultaneous marketing, media coverage, and seasonal demand obscure whether model updates truly drive engagement gains. This paper presents a novel causal estimation approach that leverages non uniform quality improvements across capabilities within a single model…