Lifelong Learning with Non-i.i.d. Tasks

In this work we aim at extending theoretical foundations of lifelong learning. Previous work analyzing this scenario is based on the assumption that the tasks are sampled i.i.d. Instead we study two scenarios when lifelong learning is possible, even though the observed tasks do not form an i.i.d. In the first case we prove a PAC-Bayesian theorem, which can be seen as a direct generalization of the analogous previous result for the i.i.d. For the second scenario we propose to learn an inductive bias in form of a transfer procedure.