Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformers

The transformer architecture introduced by [V aswani et al., 2017] has demonstrated capabilities across a wide range of tasks [Liu et al., 2018, Devlin et al., 2019, Rad-ford et al., 2019, Touvron et al., 2023, OpenAI, 2023], showing particularly significant progress in logical reasoning. These models can not only solve complex mathematical problems Davies et al. [2021] but have also reached performance levels comparable to top human contestants in the International Mathematical Olympiad (IMO) [Trinh et al., 2024]. The reasoning capabilities of large language models are fundamentally shaped by the thinking strategies they employ.