Evaluate Language Understanding of AI Models

The GLUE benchmark contains datasets and measures to evaluate general NLP models. With many general-purpose language models available today, it is important to know how they perform across different tasks and not just a specific one. There is also a leaderboard that shows the ranking of these general purpose models on different datasets. We discuss each task briefly followed by an example. Understanding some basic metrics like accuracy, F1-score would be helpful to grasp how these models are evaluated.