StepWiser: Stepwise Generative Judges for Wiser Reasoning

However, this approach suffers from two major drawbacks. First, current PRMs typically function as "black-box" classifiers, providing a score or label without explaining why a step is correct or flawed. Second, their reliance on supervised fine-tuning (SFT) with static datasets can limit their ability to generalize to new reasoning patterns (Lightman et al., 2023; Luo et al., 2024; Wang et al., 2023; Xiong et al., 2024b; Zhang et al., 2024a). We conduct a comprehensive evaluation of our method across three key dimensions: (i) the judge's To improve the reliability of multi-step reasoning in LLMs, one can consider methods beyond evaluating only the final answer, termed Outcome Reward Models (ORMs), by instead evaluating each intermediate step, a method pioneered by Process Reward Models (PRMs). Subsequent research has focused on automating this annotation process. Wang et al. (2023) proposed Concurrent work by He et al. (2025) uses a prompting approach to segment thought process into This involves replacing the language model's final layer with a linear head and fine-tuning Here, the evaluation itself is framed as a reasoning task. The judge first generates an explicit CoT to explain its rationale before outputting its final judgment.