On the Empirical Complexity of Reasoning and Planning in LLMs