Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective