A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanism