Getting Meaning from Text: Self-attention Step-by-step Video

In October 2019, Google announced that it would process search queries with the BERT model that its researchers have developed. This model can grasp difficult nuances of language: in the search 2019 brazil traveler to the USA need a visa, it is understood that the traveler is Brazilian, and the destination is the USA. From now on, Google says, this search returns the page of the U.S. embassy in Brazil, and no longer shows a page about U.S. citizens traveling to Brazil. Remarkably, most of the attention mechanism at the core of many transformer models like BERT relies on just a few basic vector operations. Let's see how it works.