Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorability
序贯可忽略性下部分可观测马尔可夫决策过程的离策略评估
We consider off-policy evaluation of dynamic treatment rules under sequential ignorability, given an assumption that the underlying system can be modeled as a partially observed Markov decision process (POMDP). We propose an estimator, partial history importance weighting, and show that it can consistently estimate the stationary mean rewards of a target policy given long enough draws from the behavior policy. We provide an upper bound on its error that decays polynomially…