Beyond Paragraphs: NLP for Long Sequences

There're many NLP problem relies on long documents. For example, Multi-Hop QA need to take many documents as input to answer the given question. Summarization may also need to take the whole document as input to extract the key part of the paragraph. Also, in the real word, existing data may be in the format of long sequence. A key challenge of long sequence NLP is that many popular algorithms such as RNN/LSTM and Transformers are focused on short sequence.