Discovering the Encoded Linguistic Knowledge in NLP Models

This article elaborates on a niche aspect of the broader cover story on "Rise of Modern NLP and the Need of Interpretability!"At Embibe, we desiderate answers to the open questions while we build the NLP platform to solve numerous problems for the academic content. Modern NLP models (BERT, GPT, etc) are typically trained in the end to end manner, carefully crafted feature engineering is now extinct, and complex architectures of these NLP models enable it to learn end-to-end tasks (e.g. Linguistic features (like part-of-speech, co-reference, etc) have played a key role in the classical NLP. Hence, it is important to understand how modern NLP models are arriving at decisions by "probing" into what all they learn. Do these models learn linguistic features from unlabelled data automatically?