Philippe Rolet, TAO, LRI, Inria Saclay-IDF, UMR CNRS 8623, Univ. Paris-Sud, France
Michèle Sebag, TAO, LRI, Inria Saclay-IDF, UMR CNRS 8623, Univ. Paris-Sud, France
Olivier Teytaud, TAO, LRI, Inria Saclay-IDF, UMR CNRS 8623, Univ. Paris-Sud, France

Links

Session:
Springer Link:

Abstract

This paper focuses on Active Learning with a limited number of queries; in application domains such as Numerical Engineering,  the size of the training set might be limited to a few dozen or hundred  examples due to computational constraints. Active Learning under bounded resources is formalized as a finite horizon Reinforcement Learning problem, where the sampling strategy aims at minimizing the expectation of the generalization error. A tractable approximation of the optimal (intractable) policy is presented, the Bandit-based Active Learner (Baal) algorithm. Viewing  Active Learning as a single-player game, Baal combines UCT,  the tree structured multi-armed bandit algorithm proposed by Kocsis and Szepesvari (2006), and billiard algorithms.  A proof of principle of the approach demonstrates its good empirical convergence toward an optimal policy and its ability to incorporate  prior AL criteria. Its hybridization with the Query-by-Committee approach is found to improve on both stand-alone Baal and stand-alone QbC.