Introduction to Sequence Modeling Problems - WebSystemer.no

In sequence learning problems, we know that the true output at timestep't' is dependent on all the inputs that the model has seen up to the time step't'. Since we don't know the true relationship, we need to come up with an approximation such that the function would depend on all the previous inputs. The key thing to note here is that the task is not changing for every timestep, whether we are predicting the next character or tagging the part of speech of a word. The input to the function is changing at every time step because for longer sentences the function needs to keep track of the larger set of words. Recurrent Neural Networks(RNN) are a type of Neural Network where the output from the previous step is fed as input to the current step.