OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning

Large language models (LLMs) [1, 8, 13] have become central to natural language processing (NLP) but their adaptation to downstream tasks remains resource-intensive. Fine-tuning these models, particularly using gradient-based methods [16, 11, 15], requires substantial memory and compute due to the overhead of backpropagation. Zeroth-order optimization (ZO) [18, 23] has emerged as a promising alternative, offering memory-efficient fine-tuning by estimating gradients through forward passes alone. This approach is especially appealing for environments with limited computational resources, such as edge devices or single-GPU setups [9]. However, despite its advantages, ZO typically exhibits slower convergence and higher variance in gradient estimates [14, 18, 7], often leading to suboptimal model performance. Prior work has explored algorithmic adjustments, including tuning hyperparameters like perturbation magnitude and sampling strategies. However, these refinements have produced only limited improvements [22, 23]. This limitation suggests a need for complementary strategies that address ZO's sensitivity to data characteristics. This paper presents a new perspective on adapting training data for ZO.