Tsuyoshi Ueno, Kyoto University, Japan
Shin-ichi Maeda, Kyoto University, Japan
Motoaki Kawanabe, Fraunhofer First, Germany
Shin Ishii, Kyoto University, Japan

Links

Session:
Springer Link:

Abstract

In this study, we extend the framework of semiparametric statistical inference introduced recently to reinforcement learning (Ueno, et.al., 2008) to online learning procedures for policy evaluation. This generalization enables us to investigate statistical properties of value function estimators both by batch and online procedures in a unified way in terms of estimating functions. Furthermore, we propose a novel online learning algorithm with optimal estimating functions which achieve the minimum estimation error. Our theoretical developments are confirmed using a simple chain walk problem.