Learning AI If You Suck at Math - Part Eight - The Musician in the Machine

"Attention takes two sentences, turns them into a matrix where the words of one sentence form the columns, and the words of another sentence form the rows, and then it makes matches, identifying relevant context." Check out the graphic from the Attention is All You Need paper below. It's two sentences, in different languages (French and English), translated by a professional human translator. The attention mechanism can generate a heat map, showing what French words the model focused on to generate the translated English words in the output.