Induction of Non-Monotonic Logic Programs to Explain Boosted Tree Models Using LIME

We present a heuristic based algorithm to induce non-monotonic logic programs that would explain the behavior of XGBoost trained classifiers. We use the LIME technique to locally select the most important features contributing to the classification decision. Then, in order to explain the model's global behavior, we propose the UFOLD algorithm ---a heuristic-based ILP algorithm capable of learning non-monotonic logic programs--- that we apply to a transformed dataset produced by LIME. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of the classification evaluation metrics. Meanwhile, the number of induced rules dramatically decreases compared ALEPH, a state-of-the-art ILP system. While the proposed approach is agnostic to the choice of ILP algorithm, our experiments suggest that the UFOLD algorithm almost always outperforms ALEPH once incorporated in this approach.