Tutorial by

Pádraig Cunningham
(University College Dublin)

Friday 11 September (Room Concordia, Hotel Golf)

Abstract:

The emphasis on evaluation is a special characteristic of Machine Learning (ML) research. Almost all ML research papers will contain a quantitative evaluation of the methods being proposed and issues with evaluation are among the most common reasons for rejecting papers or referring them for revision. At the same time papers covering ML techniques are sometimes published where the evaluation does not really establish the claims set out in the paper.
This tutorial will have two parts, the first part will cover appropriate methodologies for evaluation in ML and the second part will address common mistakes and shortcomings in evaluation. The tutorial will cover a number of practical examples and datasets for these examples can be downloaded from the tutorial web page at http://www.csi.ucd.ie/content/ecml-pkdd-2009-tutorial-evaluation. The web page also contains links to a number of research papers on evaluation in machine learning that are referenced in the tutorial.