Compressing Context to Enhance Inference Efficiency of Large Language Models