Controllable Text Generation for Large Language Models: A Survey
Liang, Xun, Wang, Hanyu, Wang, Yezhaohui, Song, Shichao, Yang, Jiawei, Niu, Simin, Hu, Jie, Liu, Dan, Yao, Shunyu, Xiong, Feiyu, Li, Zhiyu
–arXiv.org Artificial Intelligence
In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world applications, LLMs must meet increasingly complex requirements. Beyond avoiding misleading or inappropriate content, LLMs are also expected to cater to specific user needs, such as imitating particular writing styles or generating text with poetic richness. These varied demands have driven the development of Controllable Text Generation (CTG) techniques, which ensure that outputs adhere to predefined control conditions--such as safety, sentiment, thematic consistency, and linguistic style--while maintaining high standards of helpfulness, fluency, and diversity. This paper systematically reviews the latest advancements in CTG for LLMs, offering a comprehensive definition of its core concepts and clarifying the requirements for control conditions and text quality. We categorize CTG tasks into two primary types: content control and attribute control. The key methods are discussed, including model retraining, fine-tuning, reinforcement learning, prompt engineering, latent space manipulation, and decoding-time intervention. We analyze each method's characteristics, advantages, and limitations, providing nuanced insights for achieving generation control. Additionally, we review CTG evaluation methods, summarize its applications across domains, and address key challenges in current research, including reduced fluency and practicality. We also propose several appeals, such as placing greater emphasis on real-world applications in future research. This paper aims to offer valuable guidance to researchers and developers in the field. Our reference list and Chinese version are open-sourced at https://github.com/IAAR-Shanghai/CTGSurvey.
arXiv.org Artificial Intelligence
Aug-22-2024
- Country:
- Africa > Ethiopia
- Addis Ababa > Addis Ababa (0.04)
- Asia
- China
- Indonesia > Bali (0.04)
- Middle East
- Jordan (0.05)
- UAE > Abu Dhabi Emirate
- Abu Dhabi (0.04)
- Myanmar > Tanintharyi Region
- Dawei (0.04)
- Singapore (0.04)
- Taiwan > Taiwan Province
- Taipei (0.04)
- Thailand > Bangkok
- Bangkok (0.04)
- Europe
- Czechia > Prague (0.04)
- France > Auvergne-Rhône-Alpes
- Ireland > Leinster
- County Dublin > Dublin (0.04)
- Italy > Calabria
- Catanzaro Province > Catanzaro (0.04)
- Middle East > Malta
- Eastern Region > Northern Harbour District > St. Julian's (0.04)
- Romania > Sud - Muntenia Development Region
- Giurgiu County > Giurgiu (0.04)
- Spain > Catalonia
- Barcelona Province > Barcelona (0.04)
- Switzerland (0.04)
- North America
- Canada
- British Columbia > Metro Vancouver Regional District
- Vancouver (0.04)
- Ontario > Toronto (0.05)
- British Columbia > Metro Vancouver Regional District
- Dominican Republic (0.04)
- Puerto Rico > San Juan
- San Juan (0.04)
- United States
- New York > New York County
- New York City (0.04)
- Washington > King County
- Seattle (0.14)
- Massachusetts > Middlesex County
- Cambridge (0.14)
- Oregon > Multnomah County
- Portland (0.04)
- Pennsylvania > Philadelphia County
- Philadelphia (0.04)
- Hawaii > Honolulu County
- Honolulu (0.04)
- Colorado > Denver County
- Denver (0.04)
- California > San Diego County
- San Diego (0.04)
- Minnesota > Hennepin County
- Minneapolis (0.14)
- Michigan > Washtenaw County
- Ann Arbor (0.04)
- New York > New York County
- Canada
- South America > Colombia
- Meta Department > Villavicencio (0.04)
- Africa > Ethiopia
- Genre:
- Overview (1.00)
- Research Report > Promising Solution (0.46)
- Industry:
- Banking & Finance (0.92)
- Education (1.00)
- Health & Medicine
- Consumer Health (0.67)
- Therapeutic Area (0.46)
- Leisure & Entertainment (0.67)
- Technology: