Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling

Open in new window