Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression