The Algorithm Selection Competition Series 2015-17

The algorithm selection problem is to choose the most suitable algorithm for solving a given problem instance and thus, it leverages the complementarity between different approaches that is present in many areas of AI. We report on the state of the art in algorithm selection, as defined by the Algorithm Selection Competition series 2015 to 2017. The results of these competitions show how the state of the art improved over the years. Although performance in some cases is very promising, there is still room for improvement in other cases. Finally, we provide insights into why some scenarios are hard, and pose challenges to the community on how to advance the current state of the art. Keywords: 1. Introduction Algorithm Selection, Meta-Learning, Competition Analysis In many areas of AI, there are different algorithms to solve the same type of problem. Often, these algorithms are complementary in the sense that one algorithm works well when others fail and vice versa. For example in propositional satisfiability solving (SAT), there are complete tree-based solvers aimed at structured, industrial-like problems, and local search solvers aimed at randomly generated problems. In many practical cases, the performance difference between algorithms can be very large, for example as shown by Xu et al. (2012) for SAT. Per-instance algorithm selection (Rice, 1976) is a way to leverage this complementarity between different algorithms.