A Relative Ignorability Framework for Decision-Relevant Observability in Control Theory and Reinforcement Learning
控制理论与强化学习中决策相关可观测性的相对可忽略性框架
Sequential decision-making systems routinely operate with missing or incomplete data. Classical reinforcement learning theory, which is commonly used to solve sequential decision problems, assumes Markovian observability, which may not hold under partial observability. Causal inference paradigms formalise ignorability of missingness. We show these views can be unified and generalized in order to guarantee Q-learning convergence even when the Markov property fails. To do so,…