When is statistical evidence strong enough? Using hypothesis tests to value data collection
Policy decisions often hinge on conventional p-value thresholds, which ignore economic costs and benefits of further data collection. This paper recasts statistical significance as a choice between making an immediate policy recommendation and deferring it until further evidence is collected. The welfare-optimal decision corresponds, under minimax regret, to a statistical test whose level depends on the cost and precision of additional evidence. Inverting this rule, we…