Pre-Trained Language Models for Keyphrase Prediction: A Review

In the realm of NLP, BERT [2], extraction involves using a model to accurately identify GPT [3], and T5 [4] are some of the notable works that and classify the keyphrases in the document. The generation have consistently updated benchmark records in Pretrained of keyphrases is another task in which the model Language Model Keyphrase Extraction (PLM-predicts both present and absent keyphrases within the KPE) and Pre-trained Language Model Keyphrase Generation context of the document, introduced in [1]. The application (PLM-KPG) tasks [5], contributing significantly of deep learning technologies has witnessed to the development of NLP. a noticeable rise in using pre-trained language models The process of extracting keyphrases from a document (PLMs) in NLP in recent years. PLMs are trained using involves identifying and extracting significant different strategies on extensive text corpora and have phrases that represent the main topics or concepts discussed shown exceptional performance in various downstream within it. The primary objective is to extract the tasks, including Keyphrase Predation. PLMs using most essential and representative phrases using featurebased self-supervised learning differ from traditional learning [6, 7, 8, 9, 10] and linguistic techniques [11] methods, such as supervised learning, because they are like frequency analysis [12], part-of-speech tagging first trained on a large volume of unlabeled data before [13, 14], and syntactic parsing [15]. These methods fine-tuning small quantities of labeled data for specific can identify keyphrases based on their frequency, relevance, tasks.