Baidu has a new trick for teaching AI the meaning of language

BERT, by contrast, considers the context before and after a word all at once, making it bidirectional. It does this using a technique known as "masking." In a given passage of text, BERT randomly hides 15% of the words and then tries to predict them from the remaining ones. This allows it to make more accurate predictions because it has twice as many cues to work from. In the sentence "The man went to the ___ to buy milk," for example, both the beginning and the end of the sentence give hints at the missing word. The ___ is a place you can go and a place you can buy milk.