Graph based manifold regularized deep neural networks for automatic speech recognition

ABSTRACT Deep neural networks (DNNs) have been successfully applied to a wide variety of acoustic modeling tasks in recent years. These include the applications of DNNs either in a discriminative feature extraction or in a hybrid acoustic modeling scenario. Despite the rapid progress in this area, a number of challenges remain in training DNNs. This paper presents an effective way of training DNNs using a manifold learning based regularization framework. In this framework, the parameters of the network are optimized to preserve underlying manifold based relationships between speech feature vectors while minimizing a measure of loss between network outputs and targets. This is achieved by incorporating manifold based locality constraints in the objective criterion of DNNs. Empirical evidence is provided to demonstrate that training a network with manifold constraints preserves structural compactness in the hidden layers of the network. Manifold regularization is applied to train bottleneck DNNs for feature extraction in hidden Markov model (HMM) based speech recognition. The experiments in this work are conducted on the Aurora-2 spoken digits and the Aurora-4 read news large vocabulary continuous speech recognition tasks. The performance is measured in terms of word error rate (WER) on these tasks. It is shown that the manifold regularized DNNs result in up to 37% reduction in WER relative to standard DNNs. Index Terms-- manifold learning, deep neural networks, manifold regularization, manifold regularized deep neural networks, speech recognition 1. INTRODUCTION Recently there has been a resurgence of research in the area of deep neural networks (DNNs) for acoustic modeling in automatic speech recognition (ASR) [1-6]. Much of this research has been concentrated on techniques for regularization of the algorithms used for DNN parameter estimation [7-9]. At the same time, there has also been a great deal of research on graph based techniques that facilitate the preservation of local neighborhood relationships among feature vectors for parameter estimation in a number of application areas [10-13]. Algorithms that preserve these local relationships are often referred to as having the effect of applying manifold based constraints.