Audio deepfake detection (ADD) is crucial to combat the misuse of speech synthesized by generative AI models.

This paper presents anewUNIfied pre-trainedLanguageModel (UNILM)that can be fine-tuned for both natural language understanding and generation tasks.

First, a student model is trained to recover the classifier'spredictions andtomatch theexplanations givenbytheexplainer.

AwidevarietyofNLPapplications, suchasmachinetranslation, summarization, and dialog, involve text generation. One major challenge for these applications is how to evaluate whether such generated texts are actually fluent, accurate, or effective. In this work, we conceptualize theevaluation of generated text as a text generation problem, modeled using pre-trained sequence-to-sequence models. The general idea is that models trained to convert the generated text to/from a reference output or the source text will achieve higher scores when the generated text is better.

Generated text may contain offensive or toxic language, contain significant repetition, orbeofadifferent sentiment than desired by the user. We consider thetaskofunlearningthese misalignments byfine-tuning thelanguage model on signals of whatnot to do.

In this work, we introduce WinoGAViL: an online game of vision-andlanguage associations (e.g., betweenwerewolves and a full moon), used as a dynamic evaluation benchmark.

In order to explore the impact of pandemics on people's lives, itiscrucial to understand the public'sconcerns and attitudes towards pandemicrelated entities (e.g., drugs, vaccines) onsocial media.

Example Reweightingdirectly assigns an importance weight to the standard CE training loss, accordingtothebiasdegreeβ: Lreweight = (1 β)y logpm (3) Confidence Regularizationis based on knowledge distillation [9]. It involves a teacher model trainedwiththestandardCEloss. Specifically, we calculate the weighted average of the F1 score of each class. The splits used for evaluation are highlightedwithredcolor. To address this problem, we select the best checkpoint after0.7 tmax of training, butstill according to the performance on the ID devset.

We introduce Lifelong ICL, a problem setting that challenges long-context language models (LMs) to learn a sequence of language tasks through in-context learning(ICL).