Tackling multiple tasks with a single visual language model

One key aspect of intelligence is the ability to quickly learn how to perform a new task when given a brief instruction. But for a typical visual model to learn a new task, it must be trained on tens of thousands of examples specifically labelled for that task. If the goal is to count and identify animals in an image, as in "three zebras", one would have to collect thousands of images and annotate each image with their quantity and species. This process is inefficient, expensive, and resource-intensive, requiring large amounts of annotated data and the need to train a new model each time it's confronted with a new task. As part of DeepMind's mission to solve intelligence, we've explored whether an alternative model could make this process easier and more efficient, given only limited task-specific information.