Hierarchical Reasoning Models: Perspectives and Misconceptions

Transformers have demonstrated remarkable performance in natural language processing and related domains, as they largely focus on sequential, autoregressive next-token prediction tasks. An emerging exploration in this direction is the Hierarchical Reasoning Model Wang et al. (2025a), which introduces a novel type of recurrent reasoning in the latent space of transformers, achieving remarkable performance on a wide range of 2D reasoning tasks. Despite the promising results, this line of models is still at an early stage and calls for in-depth investigation. In this work, we review this class of models, examine key design choices, test alternative variants and clarify common misconceptions. The Transformer architecture (V aswani et al., 2017) has become the foundation of modern large language models (LLMs), powering systems such as BERT (Devlin et al., 2019), PaLM (Chowdhery et al., 2022) and GPT series (Brown et al., 2020; Achiam et al., 2023).