Olga Kozlova, Thales Security Solutions & Services, Simulation, France
Olivier Sigaud, Universite Pierre et Marie Curie - Paris 6, CNRS UMR 7222, France
Pierre-Henri Wuillemin, Universite Pierre et Marie Curie - Paris 6, CNRS UMR 7606, France
Christophe Meyer, Thales Security Solutions & Services, ThereSIS Research and Innovation Office, France

Links

Session:
Springer Link:

Abstract

The Factored Markov Decision Process (FMDP) framework is a standard representation for sequential decision problems under uncertainty where the state is represented as a collection of random variables. Factored Reinforcement Learning (FRL) is an Model-based Reinforcement Learning approach to FMDPs where the transition and reward  unctions of the problem are learned. In this paper, we show how to model in a theoretically well-founded way the problems where some combinations of state variable values may not occur, giving rise to impossible states. Furthermore, we propose a new heuristics that considers as impossible the states that have not been seen so far. We derive an algorithm whose improvement in performance with respect to the standard approach is illustrated through benchmark experiments.