Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models