Mutual information, metric entropy and cumulative relative entropy risk
互信息、度量熵与累积相对熵风险
Assume ${P_{\theta}: \theta \epsilon \Theta}$ is a set of probability distributions with a common dominating measure on a complete separable metric space Y. A state $\theta^* \epsilon \Theta$ is chosen by Nature. A statistician obtains n independent observations $Y_1, \dots, Y_n$ from Y distributed according to $P_{\theta^*}$. For each time t between 1 and n, based on the observations $Y_1, \dots, Y_{t-1}$, the statistician produces an estimated distribution $\hat{P}_t$ for $P_{\theta^*}$…