Google's Reformer: An Important Breakthrough of 2020

Regardless of whether it's language, music, speech, or video, sequential data isn't simple for AI and machine learning models to understand, especially when it relies upon the extensive surrounding context. For example, if an individual or an item vanishes from view in a video just to return a lot later, numerous algorithms will overlook what it looked like. Researchers at Google set out to solve this with Transformer, a design that reached out to thousands of words, drastically improving performance in tasks like song composition, image synthesis, sentence-by-sentence text translation, and document summarization. In any case, Transformer isn't flawless by any stretch, extending it to bigger contexts makes clear its restrictions. Applications that utilize enormous windows have memory necessities going from gigabytes to terabytes in size, which means models can just ingest a few paragraphs of text or create short bits of music.