Understanding the Outputs of Multi-Layer Bi-Directional LSTMs

In the world of machine learning, long short-term memory networks (LSTMs) are a powerful tool for processing sequences of data such as speech, text, and video. The recurrent nature of LSTMs allows them to remember pieces of data that they have seen earlier in the sequence. Conceptually, this is easier to understand in the forward direction (i.e., start to finish), but it can also be useful to consider the sequence in the opposite direction (i.e., finish to start). Joe likes blank, especially if they're fried, scrambled, or poached. In the forward direction, the only information available before reaching the missing word is "Joe likes blank ", which could have any number of possibilities.