End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results

Dzmitry Bahdanau Jacobs University Bremen, Germany Yoshua Bengio Université de Montréal CIFAR Senior Fellow We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bidirectional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes. The alignment between the input and output sequences is established using an attention mechanism: the decoder emits each symbol based on a context created with a subset of input symbols selected by the attention mechanism. We report initial results demonstrating that this new approach achieves phoneme error rates that are comparable to the state-of-the-art HMM-based decoders, on the TIMIT dataset.