LAB: Large-Scale Alignment for ChatBots

This work introduces LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training. Leveraging a taxonomy-guided synthetic data generation process and a multi-phase tuning framework, LAB significantly reduces reliance on expensive human annotations and proprietary models like GPT-4. We demonstrate that LAB-trained models can achieve competitive performance across several benchmarks compared to models trained with traditional human-annotated or GPT-4 generated synthetic data. Large language models (LLMs) have achieved remarkable levels of success in various natural language processing (NLP) applications, including question-answering, entity extraction, and summarization . This has been made possible, in large part, by the introduction of the transformer architecture, which can leverage large amounts of unlabeled, unstructured data, enabling the scaling of LLMs to billions, or even trillions of parameters. LLMs are typically trained in phases: a self-supervised pre-training phase, followed by supervised alignment tuning phases.