Rectifying LLM Thought from Lens of Optimization