TransformerMemoryasa DifferentiableSearchIndex

This proposal is shown in the bottom half of Figure 1, for a sequence-to-sequence encoder-decoder architecture. We call this proposed architecture adifferentiable search index(DSI), and implement it with a largepre-trained Transformer (Vaswanietal.,2017)model,