Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference
冰淇淋不会导致溺水：评测大语言模型应对因果推断统计陷阱的能力AI 解读 ↓
← 返回论文列表与筛选
阅读说明
本页整理公开书目信息，不生成学术质量评分。AI 解读根据题名及所注明的摘要或原文片段生成，未完成全文核验，可能有误；具体假设、证明与结论请核对原文。摘要保留来源文本，未经模型改写。
DOI 首次登记不等于接收或发表日期。部分更正、评论等条目没有独立摘要；缺少可靠材料时保留待补状态。
本页为预印本版本；arXiv 收录、作者填写的期刊信息以及作者既有发表经历，都不单独证明本篇已接收。作者发表记录按 OpenAlex 身份与完整姓名、本站覆盖期刊 ISSN 及正式卷期核对，只展示已核实例，档案仍可能误合并。
引用量采用 OpenAlex 单条关联记录，不合计多个记录；关联记录可能包含正式发表版本，不等于此版本单独引用。缺失不视为零，引用指标不代表论文质量。
关注作者按书目中的完整姓名匹配，同名和姓名变体需人工辨别。相关论文先按共享细分方向和具体方法排列，再参考通用方法与发表日期。
分类依据与纠错
AI 根据所提供的论文材料归类：摘要以因果推断中的混杂、选择偏差和统计推理陷阱作为评测对象。
反馈分类问题
正在核对关联版本…
代码、数据与补充材料
正在核对对应版本的资源…
冰淇淋不会导致溺水：评测大语言模型应对因果推断统计陷阱的能力
- 研究问题
- 大语言模型能否在不同难度的任务中识别并处理辛普森悖论、选择偏差等统计因果陷阱，其回答与自动评分是否可靠？
- 方法与思路
- 作者提出配有分级挑战和评分标准的 CausalPitfalls 基准，以直接提示及代码辅助提示评测模型，并将评分与人类专家判断比较。
- 材料陈述的贡献
- 摘要称现有模型在统计因果推断中存在明显局限，所提基准为因果推理能力和回答可靠性提供量化指标，并检验评分机制的有效性。
- 阅读关注点
- 应核对题目构造与真实统计难点的对应、数据或题目泄漏风险、代码执行与评分流程，以及专家一致性和模型版本的记录。
- 阅读边界
- 这是 arXiv 预印本，同行评审及期刊接收未核实；摘要未给模型名单、评分数值或专家一致性，不能据此评价任一具体模型的能力。
原文摘要
Reliable causal inference is essential for making decisions in high-stakes areas like medicine, economics, and public policy. However, it remains unclear whether large language models (LLMs) can handle rigorous and trustworthy statistical causal inference. Current benchmarks usually involve simplified tasks. For example, these tasks might only ask LLMs to identify semantic causal relationships or draw conclusions directly from raw data. As a result, models may overlook important statistical pitfalls, such as Simpson's paradox or selection bias. This oversight limits the applicability of LLMs in the real world. To address these limitations, we propose CausalPitfalls, a comprehensive benchmark designed to rigorously evaluate the capability of LLMs in overcoming common causal inference pitfalls. Our benchmark features structured challenges across multiple difficulty levels, each paired with grading rubrics. This approach allows us to quantitatively measure both causal reasoning capabilities and the reliability of LLMs' responses. We evaluate models using two protocols: (1) direct prompting, which assesses intrinsic causal reasoning, and (2) code-assisted prompting, where models generate executable code for explicit statistical analysis. Additionally, we validate the effectiveness of this judge by comparing its scoring with assessments from human experts. Our results reveal significant limitations in current LLMs when performing statistical causal inference. The CausalPitfalls benchmark provides essential guidance and quantitative metrics to advance the development of trustworthy causal reasoning systems.