How to set AdamW's weight decay as you scale model and dataset size

Open in new window