One model to learn them all

I recently stumbled upon this paper by the Google Brain team called "One model to learn them all". I just had to open it and take a look at the paper because of its great title (best ML paper title ever?). I quickly discovered that it is about a very fascinating and interesting idea. In this article I want to quickly summarize what I personally found the most interesting. Can we create a unified deep learning model to solve tasks across multiple domains?