Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning

The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the runtime.