Recent Advances in Conversational Speech Recognition

Our second model, called very deep convolutional neural net (or CNN), has its origins in image classification [4]. Speech can be viewed as an image if we consider the spectral representation of the audio signal with the two dimensions being time and frequency. As opposed to the classic CNN architectures employed in our previous system [5] that have only one or two convolutional layers with large (typically 9-by-9) kernels, our very deep CNN [6] has up to ten convolutional layers with small 3-by-3 kernels which preserve the dimensionality of the input. By stacking many of these convolutional layers with Rectified Linear Units nonlinearities before pooling layers, the same receptive field is created with less parameters and more nonlinearity. These two models which differ radically in architecture and input representation show good complementarity and their combination leads to additional gains over the best individual model.