In-Depth Look at Word Filling Societal Bias Measures

We propose a way to improve the methodologies by introducing a new Language models (LMs) are ubiquitous in current score definition. During experiments we introduce NLP and have brought undeniable performance improvements several new variants of the existing datasets and for many tasks. Concerns have been a completely new dataset in Slovak. These new raised about the fairness of these models (Blodgett datasets are used to compare the expected behavior et al., 2020; Shah et al., 2020; Dev et al., 2021b). of the LMs with their actual behavior. Since LMs are usually trained with web-based text Our results challenge the validity of previous corpora generated by a general population, there studies.