RNNs in Tensorflow, a Practical Guide and Undocumented Features

In a previous tutorial series I went over some of the theory behind Recurrent Neural Networks (RNNs) and the implementation of a simple RNN from scratch. That's a useful exercise, but in practice we use libraries like Tensorflow with high-level primitives for dealing with RNNs. With that using an RNN should be as easy as calling a function, right? In this post I want to go over some of the best practices for working with RNNs in Tensorflow, especially the functionality that isn't well documented on the official site. Check out the tf.SequenceExample Jupyter Notebook here! RNNs are used for sequential data that has inputs and/or outputs at multiple time steps.