Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture

Yet, existing low-resource learning techniques, such as Active Learning (AL), that aim to support human annotators mostly focus on the label while neglecting the natural language explanation of a data point. This work proposes a novel AL architecture to support experts' real-world need for label and explanation annotations in low-resource scenarios. Our AL architecture leverages an explanationgeneration model to produce explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a novel data diversity-based AL sampling strategy that benefits from the explanation annotations. Automated and human evaluations demonstrate the effectiveness of incorporating explanations Figure 1: Our dual-model AL system architecture at into AL sampling and the improved human annotation every iteration: 1) the AL data selector chooses a few efficiency and trustworthiness with our unlabeled examples; 2) human annotators provide an AL architecture. Additional ablation studies illustrate explanation and label for each data instance; 3) the annotated the potential of our AL architecture explanations are used to finetune the explanationgeneration for transfer learning, generalizability, and integration model; 4) the annotated labels and generated with large language models (LLMs).