A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning

Multi-task learning (MTL) is broadly used across various applications of machine learning and has several advantages in comparison with the single-task learning. Since layers are shared between different tasks and features are not repeatedly calculated for each task, the amount of memory used is reduced and the inference speed is improved. In addition, if tasks share complimentary information, they act as regularizers for each other which results in the improvement of the prediction performance of each task [1]. This has been proven in various areas such as detection and classification [2], computer vision [3, 4], depth estimation [5], natural language processing [6-8] and drug discovery [9]. The goal of this review paper is to provide an overview of various deep multi-task learning (dMTL) solutions and possible improvements in performance through efficient auxiliary tasks selection.