On the O(d K1/4)Convergence Rate of AdamW Measured by ℓ1 Norm

As the default optimizer for training large language models, AdamW has achieved remarkable success in deep learning. However, its convergence behavior is not theoretically well-understood.