Recent advances in Chain-of-Thought (CoT) prompting have substantially improved the reasoning capabilities of Large Language Models (LLMs). However, these methods often suffer from overthinking, leading to unnecessarily lengthy or redundant reasoning traces. Existing approaches attempt to mitigate this issue through curating multiple reasoning chains for training LLMs, but their effectiveness is often constrained by the quality of the generated data and prone to overfitting. To address the challenge, we propose Reasoning Compression ThroUgh Stepwise Trials (ReCUT), a novel method aimed at balancing the accuracy and length of reasoning trajectory. Specifically, ReCUT employs a stepwise exploration mechanism and a long-short switched sampling strategy, enabling LLMs to incrementally generate diverse reasoning paths. These paths are evaluated and used to construct preference pairs to train two specialized models (Gemini LLMs)-one optimized for reasoning accuracy, the other for shorter reasoning. A final integrated model is obtained by interpolating the parameters of these two models. Experimental results across multiple math reasoning datasets and backbone models demonstrate that ReCUT significantly reduces reasoning lengths by approximately 30-50%, while maintaining or improving reasoning accuracy compared to various baselines. All codes and data will be released via https://github.com/NEUIR/ReCUT.

Existing Large Multimodal Models (LMMs) struggle with mathematical geometric reasoning due to a lack of high-quality image-text paired data. Current geometric data generation approaches, which apply preset templates to generate geometric data or use Large Language Models (LLMs) to rephrase questions and answers (Q&A), unavoidably limit data accuracy and diversity. To synthesize higherquality data, we propose a two-stage Reverse Chain-of-Thought (R-CoT) geometry problem generation pipeline. First, we introduce GeoChain to produce highfidelity geometric images and corresponding descriptions highlighting relations among geometric elements. We then design a Reverse A&Q method that reasons step-by-step based on the descriptions and generates questions in reverse from the reasoning results. Experiments demonstrate that the proposed method brings significant and consistent improvements on multiple LMM baselines, achieving new performance records in the 2B, 7B, and 8B settings. Notably, R-CoT-8B significantly outperforms previous state-of-the-art open-source mathematical models by 16.6% on MathVista and 9.2% on GeoQA, while also surpassing the closedsource model GPT-4o by an average of 13% across both datasets. The code is available at https://github.com/dle666/R-CoT. Large Language Models (LLMs) exhibit excellent reasoning capabilities and draw extensive attention from the artificial intelligence research community (Lu et al., 2023b) to mathematical problemsolving in textual form (Chen et al., 2024b; Liao et al., 2024; Zhou et al., 2024; Zhao et al., 2024b; Zhou & Zhao, 2024; Kim et al., 2024). However, LLMs still struggle to solve mathematical problems involving images that require visual comprehension. Geometry problems, as typical mathematical problems with images, play an important role in evaluating mathematical reasoning skills (Zhang et al., 2023c), requiring a high level of visual comprehension. Besides, even though some problems are not related to geometry on the surface, they require the same skills for models (e.g., fine-grained image comprehension skills and multi-step reasoning skills). With the appearance of o1 (OpenAI, 2024), GPT-4o (Islam & Moushi, 2024), Gemini (Team et al., 2023), and numerous Large Multimodal Models (LMMs) (Li et al., 2024a; Liu et al., 2024; Chen et al., 2024d; Bai et al., 2023), recent researches progressively investigate using LMMs to solve mathematical geometry problems. Although LMMs show impressive results in general visual question-answering (VQA) tasks (Fan et al., 2024; Liu et al., 2024), they still face challenges in solving mathematical geometry problems. Adjust values in the question and generate answers.

LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks. Our research embarks on a quest to uncover the root causes of compositional reasoning failures of LLMs, uncovering that most of them stem from the improperly generated or leveraged implicit reasoning results. Inspired by our empirical findings, we resort to Logit Lens and an intervention experiment to dissect the inner hidden states of LLMs. This deep dive reveals that implicit reasoning results indeed surface within middle layers and play a causative role in shaping the final explicit reasoning results. Our exploration further locates multi-head self-attention (MHSA) modules within these layers, which emerge as the linchpins in accurate generation and leveraing of implicit reasoning results. Grounded on the above findings, we develop CREME, a lightweight method to patch errors in compositional reasoning via editing the located MHSA modules. Our empirical evidence stands testament to CREME's effectiveness, paving the way for autonomously and continuously enhancing compositional reasoning capabilities in language models.

In e-commerce portals, generating answers for product-related questions has become a crucial task. In this paper, we focus on the task of product-aware answer generation, which learns to generate an accurate and complete answer from large-scale unlabeled e-commerce reviews and product attributes. However, safe answer problems pose significant challenges to text generation tasks, and e-commerce question-answering task is no exception. To generate more meaningful answers, in this paper, we propose a novel generative neural model, called the Meaningful Product Answer Generator (MPAG), which alleviates the safe answer problem by taking product reviews, product attributes, and a prototype answer into consideration. Product reviews and product attributes are used to provide meaningful content, while the prototype answer can yield a more diverse answer pattern. To this end, we propose a novel answer generator with a review reasoning module and a prototype answer reader. Our key idea is to obtain the correct question-aware information from a large scale collection of reviews and learn how to write a coherent and meaningful answer from an existing prototype answer. To be more specific, we propose a read-and-write memory consisting of selective writing units to conduct reasoning among these reviews. We then employ a prototype reader consisting of comprehensive matching to extract the answer skeleton from the prototype answer. Finally, we propose an answer editor to generate the final answer by taking the question and the above parts as input. Conducted on a real-world dataset collected from an e-commerce platform, extensive experimental results show that our model achieves state-of-the-art performance in terms of both automatic metrics and human evaluations. Human evaluation also demonstrates that our model can consistently generate specific and proper answers.

This paper shows a novel fuzzy approximate reasoning method based on the least common multiple (LCM). I ts fundamental idea is to obtain a new fuzzy reasoning result by the extended distance measure based on LCM between the antecedent fuzzy set and the consequent one in discrete SISO fuzzy system. T he proposed metho d is called LCM one. And then this paper analyzes its some properties, i.e., the reductive property, information loss occurred in reasoning process, and the convergence of fuzzy control. Theoretical and experimental research results highlight that proposed method meaningfully improve the reductive property and information loss and controllability than the previous fuzzy reasoning methods.

This paper presents an original method of fuzzy approximate reasoning that can open a new direction of research in the uncertainty inference of Artificial Intelligence(AI) and Computational Intelligence(CI). Fuzzy modus ponens (FMP) and fuzzy modus tollens(FMT) are two fundamental and basic models of general fuzzy approximate reasoning in various fuzzy systems. And the reductive property is one of the essential and important properties in the approximate reasoning theory and it is a lot of applications. This paper suggests a kind of extended distance measure (EDM) based approximate reasoning method in the single input single output(SISO) fuzzy system with discrete fuzzy set vectors of different dimensions. The EDM based fuzzy approximate reasoning method is consists of two part, i.e., FMP-EDM and FMT-EDM. The distance measure based fuzzy reasoning method that the dimension of the antecedent discrete fuzzy set is equal to one of the consequent discrete fuzzy set has already solved in other paper. In this paper discrete fuzzy set vectors of different dimensions mean that the dimension of the antecedent discrete fuzzy set differs from one of the consequent discrete fuzzy set in the SISO fuzzy system. That is, this paper is based on EDM. The experimental results highlight that the proposed approximate reasoning method is comparatively clear and effective with respect to the reductive property, and in accordance with human thinking than existing fuzzy reasoning methods.

In this paper, we propose a new fuzzy reasoning principle, so called Movement and Transformation Principle(MTP). This Principle is to obtain a new fuzzy reasoning result by Movement and Transformation the consequent fuzzy set in response to the Movement, Transformation, and Movement-Transformation operations between the antecedent fuzzy set and fuzzificated observation information. And then we presented fuzzy modus ponens and fuzzy modus tollens based on MTP. We compare proposed method with Mamdani fuzzy system, Sugeno fuzzy system, Wang distance type fuzzy reasoning method and Hellendoorn functional type method. And then we applied to the learning experiments of the fuzzy neural network based on MTP and compared it with the Sugeno method. Through prediction experiments of fuzzy neural network on the precipitation data and security situation data, learning accuracy and time performance are clearly improved. Consequently we show that our method based on MTP is computationally simple and does not involve nonlinear operations, so it is easy to handle mathematically.