In Named Entity Recognition (NER), pre-trained language models have been overestimated by focusing on dataset biases to solve current benchmark datasets. However, these biases hinder generalizability which is necessary to address real-world situations such as weak name regularity and plenty of unseen mentions. To alleviate the use of dataset biases and make the models fully exploit data, we propose a debiasing method that our bias-only model can be replaced with a Pointwise Mutual Information (PMI) to enhance generalization ability while outperforming an in-domain performance. Our approach enables to debias highly correlated word and labels in the benchmark datasets; reflect informative statistics via subword frequency; alleviates a class imbalance between positive and negative examples. For long-named and complex-structure entities, our method can predict these entities through debiasing on conjunction or special characters. Extensive experiments on both general and biomedical domains demonstrate the effectiveness and generalization capabilities of the PMI.

This paper presents a discovery that the length of the entities follows a family of scale-free power law distributions. The concept of entity here broadly includes the named entity, entity mention, time expression, and domain-specific entity that are well investigated in natural language processing and related areas. The power law distributions in entity length have well-defined means and finite variances and possess the scale-free property. We explain the phenomenon of power laws in entity length by the principle of least effort in communication and the preferential mechanism.