TaxoKnow: Taxonomy as Prior Knowledge in the Loss Function of Multi-class Classification

Chiriatti 2020), have made significant advances in Natural Language Processing (NLP). In general, pre-training, where a model first trains on massive amounts of data before being fine-tuned for a specific task, has proven to be assumption in the real world. Moreover, compared to human an efficient technique for improving the performance of a capabilities, DNNs still lack in various aspects, such wide range of language tasks (Min et al. 2021). For example, as Adaptability, Generalizability, Robustness, Explainability, BERT (Devlin et al. 2018) is a pre-trained transformerbased Abstraction, Common sense, and Causal reasoning. In encoder model that can be fine-tuned for various NLP general, Multi-Layer Perceptrons (MLPs) are good at generalizing tasks, such as sentence classification, question answering, within the space of training examples, but they perform and named entity recognition. In fact, large language models poorly at generalizing outside the space of training examples, have shown a so-called few-shot learning capability to be and this limitation is not improved even by adding efficiently adapted to downstream tasks.