From complexity to simplicity ... • r/MachineLearning

I am currently working on nlp and this is another research work that I encountered - https://arxiv.org/abs/1706.03762 It's the "Attention is all you need" paper. I am pretty sure most of the folks would have already read it (reddit:)). With this one in the light now, what do you think? Is it possible that the fully connected neural network is itself the key?