Unsupervised Pre-training for Speech Recognition (wav2vec)

Deep learning model breaks through lots of state-of-the-art records in many fields which includes computer vision (CV), natural language processing (NLP) and automatic speech recognition (ASR). In CV, we can use pre-trained R-CNN, YOLO model on our target domain problem. In NLP, we can also leverage pre-trained model such as BERT and XLNet. In ASR, we now have a pre-trained model to convert audio input to a vectors. In the previous stories, we went through classic methods and Speech2vec to learn vector representations for audio inputs.