d8e1344e27a5b08cdfd5d027d9b8d6de-AuthorFeedback.pdf

The purpose ofthis is to scale down the logits before softmax is applied, a technique similar to the one seen in Vaswani et al. (2017). We will correct the typos in the paper for the camera-ready version. We understand your criticism with respect to ablation studies. Attention is all you need.