Deepmind's hunger for data: large AI models are far from being fed up

Are giant AI language models like GPT-3 or PaLM under-trained? A Deepmind study shows that we can expect further leaps in performance. Big language models like OpenAI's GPT-3, Deepmind's Gopher, or most recently Google's powerful PaLM rely on lots of data and gigantic neural networks with hundreds of billions of parameters. PaLM, with 540 billion parameters, is the largest language model to date. The trend toward more and more parameters stems from the previous finding that the capabilities of large AI models scale with their size.