Relational Learning for Collective Classification of Entities in Images

We consider the problem of discrete multi-label entity classification in images. We argue that the framework of Markov Logic can provide a unified, well-grounded mechanism to incorporate arbitrary logical relationships between entities to improve classification in images, and thus generalizes much of the recent work on exploiting local and global context in object recognition and scene understanding. Furthermore, we show that Markov Logic can provide a powerful new set of contexts that can relate entities across images in a database for joint classification of all entities in a test set simultaneously. We relate this collective classification of images to graph-based semi-supervised learning approaches, and show that Markov Logic can effectively provide a method to unify context-related work with semi-supervised approaches in a way that neither techniques could easily do on their own. Finally, we show the efficacy of these techniques on a face recognition task on three datasets showing that adding contextual relations dramatically improves accuracy over semi-supervised learning approaches alone.