Brief Review -- Chinchilla: Training Compute-Optimal Large Language Models

On all subsets, Chinchilla outperforms Gopher. On this benchmark, Chinchilla significantly outperforms Gopher despite being much smaller, with an average accuracy of 67.6% (improving upon Gopher by 7.6%). Chinchilla outperforms Gopher by 7.6% on average, performing better on 51/57 individual tasks, the same on 2/57, and worse on only 4/57 tasks. On RACE-h and RACE-m, Chinchilla considerably improves performance over Gopher. On LAMBADA, Chinchilla outperforms both Gopher and MT-NLG 530B.