A Graphical Approach to State Variable Selection in Off-policy Learning
离策略学习中状态变量选择的图方法AI 解读 ↓
← 返回论文列表与筛选
阅读说明
本页整理公开书目信息，不生成学术质量评分。AI 解读根据题名及所注明的摘要或原文片段生成，未完成全文核验，可能有误；具体假设、证明与结论请核对原文。摘要保留来源文本，未经模型改写。
DOI 首次登记不等于接收或发表日期。部分更正、评论等条目没有独立摘要；缺少可靠材料时保留待补状态。
本页为预印本版本；arXiv 收录、作者填写的期刊信息以及作者既有发表经历，都不单独证明本篇已接收。作者发表记录按 OpenAlex 身份与完整姓名、本站覆盖期刊 ISSN 及正式卷期核对，只展示已核实例，档案仍可能误合并。
引用量采用 OpenAlex 单条关联记录，不合计多个记录；关联记录可能包含正式发表版本，不等于此版本单独引用。缺失不视为零，引用指标不代表论文质量。
关注作者按书目中的完整姓名匹配，同名和姓名变体需人工辨别。相关论文先按共享细分方向和具体方法排列，再参考通用方法与发表日期。
分类依据与纠错
AI 根据所提供的论文材料归类：摘要明确处理非随机历史数据中的策略效应识别、混杂控制和因果图准则，直接支持因果推断分类，应用为序贯决策。
反馈分类问题
正在核对关联版本…
代码、数据与补充材料
正在核对对应版本的资源…
关注这篇论文的作者
离策略学习中状态变量选择的图方法
- 研究问题
- 利用非随机历史数据学习序贯决策策略时，如何选择状态变量以识别目标策略的影响，并兼顾混杂控制和降维需求？
- 方法与思路
- 作者利用无环有向混合图的因果推断理论，为涵盖动态治疗方案和马尔可夫决策过程的一般决策过程提供图形识别准则。
- 材料陈述的贡献
- 摘要称，该框架连接动态治疗方案与离线强化学习，澄清相关因果假设及误解；物流动态定价模拟显示违反准则可能产生次优策略。
- 阅读关注点
- 应核对图准则的适用决策过程、状态选择与混杂控制的关系，以及准则是否充分或必要和模拟中违反条件的具体方式。
- 阅读边界
- 依据 arXiv 预印本摘要，不能确认同行评审或期刊接收；摘要未列出图准则和完整假设，仿真表现不能确认实际物流政策收益。
原文摘要
Sequential decision problems are widely studied across many areas of science. A key challenge when learning policies from historical data - a practice commonly referred to as off-policy learning - is how to ``identify'' the impact of a policy of interest when the observed data are not randomized. Off-policy learning has mainly been studied in two settings: dynamic treatment regimes (DTRs), where the focus is on controlling confounding in medical problems with short decision horizons, and offline reinforcement learning (RL), where the focus is on dimension reduction in closed systems such as games. The gap between these two well studied settings has limited the wider application of off-policy learning to many real-world problems. Using the theory for causal inference based on acyclic directed mixed graph (ADMGs), we provide a set of graphical identification criteria in general decision processes that encompass both DTRs and MDPs. We discuss how our results relate to the often implicit causal assumptions made in the DTR and RL literatures and further clarify several common misconceptions. Finally, we present a realistic simulation study for the dynamic pricing problem encountered in container logistics, and demonstrate how violations of our graphical criteria can lead to suboptimal policies.