Deep introduction to LSTMs

We define a set of inputs (x(1), x(2), …, x(m)) and each of these numbers is going to multiplied by a weight matrix and after that, they all are going to be added together to form this internal state of the perceptron that is z. With the Perceptron, we could have multiple inputs coming in and since we're interested here in sequence modeling, we could think of these inputs as being from a single time-step from our sequence. We could also think of extending a single perceptron to a layer of perceptrons to yield multi-dimensional outputs. We know that our output vector y_hat at a particular time-step t is just going to be a function of the input at that time-step. But if we're considering sequential data, it's probably very likely that the output or the label at a later time-step is going to somehow depend on the inputs at prior time-steps so what we're missing here by treating these individual time-steps as individual isolated time-steps is this relationship that's inherent to sequence data between inputs earlier on in the sequence to what we predict later on in the sequence.