GRAIL: Gradient-Based Adaptive Unlearning for Privacy and Copyright in LLMs

Ju-Min Han Seong-Whan Lee* Dept. of Artificial Intelligence Dept. of Artificial Intelligence Korea University, Seoul, South Korea Korea University, Seoul, South Korea juminhan@korea.ac.kr sw.lee@korea.ac.kr Abstract --Large Language Models (LLMs) trained on extensive datasets often learn sensitive information, which raises significant social and legal concerns under principles such as the "Right to be forgotten." Retraining entire models from scratch to remove undesired information is both costly and impractical. T o tackle these issues, we propose GRAIL (GRadient-based AdaptIve unLearning), a novel multi-domain unlearning framework. GRAIL leverages gradient information from multiple domains to precisely distinguish the unlearning scope from the retention scope, and applies an adaptive parameter-wise localization strategy to selectively remove targeted knowledge while preserving critical parameters for each domain. Experimental results on unlearning benchmarks show that GRAIL achieves unlearning success on par with the existing approaches, while also demonstrating up to 17% stronger knowledge retention success compared to the previous state-of-art method. Our findings establish a new paradigm for effectively managing and regulating sensitive information in large-scale pre-trained language models. I NTRODUCTION Recently, Large Language Models (LLMs) [1]-[3] have been trained on extensive datasets that include web pages and user-generated content.