APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation

Adapting large pre-trained language models to specific tasks requires balancing performance improvement with preservation of learned capabilities. Standard fine-tuning approaches optimize a single objective function through gradient descent, often leading to catastrophic forgetting [16] or instability in learned representations. Parameter-efficient methods like LoRA [11] constrain modifications to low-dimensional subspaces but limit adaptation scope. We propose Adjacent Possible Exploration (APE), a selective fine-tuning approach that explores multiple parameter modification directions while implementing acceptance criteria to maintain model stability. The method draws conceptual inspiration from evolutionary optimization principles, particularly the biological constraint that viable changes must preserve essential system properties while enabling incremental improvement. APE operates by generating multiple candidate parameter updates through fine-tuning on randomly sampled data subsets, then selecting only those updates that exceed a performance improvement threshold. This creates a filtered optimization process that systematically explores beneficial parameter modifications while rejecting changes that fall within noise levels or potentially destabilize learned representations. Our key contributions include: (1) A practical algorithm for selective fine-tuning that balances exploration and stability, (2) Empirical validation showing superior performance compared to standard adaptation methods, and (3) Analysis of why selective acceptance of parameter modifications leads to more robust model adaptation. 1 The approach demonstrates that systematic exploration of parameter space through filtered selection can achieve better adaptation results than unconstrained optimization, providing a principled framework for controlled model modification that maintains stability while enabling significant performance improvements.