Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors