hipo
HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs
Deng, Ken, Zhan, Zizheng, Xiang, Wen, Zhu, Wenqiang, Li, Weihao, Xu, Jingxuan, Peng, Tianhao, Lei, Xinping, Wu, Kun, Yao, Yifan, Huang, Haoyang, Tang, Huaixi, Lei, Kepeng, Lai, Zhiyi, Yu, Songwei, Feng, Zongxian, Gao, Zuchen, Xie, Weihao, Zhang, Chenchen, Wu, Yanan, Zhang, Yuanxing, Huang, Lecheng, Zhang, Yuqun, Liu, Jie, Zhang, Zhaoxiang, Zhang, Haotian, Chen, Bin, Liu, Jiaheng
Large Language Models (LLMs) increasingly rely on Chain-of-Thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings.
Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering
Hu, Yinghao, Gan, Leilei, Xiao, Wenyi, Kuang, Kun, Wu, Fei
Hallucination, or the generation of incorrect or fabricated information, remains a critical challenge in large language models (LLMs), particularly in high-stake domains such as legal question answering (QA). In order to mitigate the hallucination rate in legal QA, we first introduce a benchmark called LegalHalBench and three automatic metrics to evaluate the common hallucinations when LLMs answer legal questions. We then propose a hallucination mitigation method that integrates behavior cloning and a novel Hard Sample-aware Iterative Direct Preference Optimization (HIPO). We conduct extensive real-data experiments to validate the effectiveness of our approach. Our results demonstrate remarkable improvements in various metrics, including the newly proposed Non-Hallucinated Statute Rate, Statute Relevance Rate, Legal Claim Truthfulness, as well as traditional metrics such as METEOR, BERTScore, ROUGE-L, and win rates.
A High-frequency Pneumatic Oscillator for Soft Robotics
Li, Longchuan, He, Shuqian, Qi, Qiukai, Cui, Ye, Yan, Cong, Jiang, Kaige, Kang, Shuai, Tokuda, Isao T., Wang, Zhongkui, Ma, Shugen, Liu, Huaping
Soft robots, while highly adaptable to diverse environments through various actuation methods, still face significant performance boundary due to the inherent properties of materials. These limitations manifest in the challenge of guaranteeing rapid response and large-scale movements simultaneously, ultimately restricting the robots' absolute speed and overall efficiency. In this paper, we introduce a high-frequency pneumatic oscillator (HIPO) to overcome these challenges. Through a collision-induced phase resetting mechanism, our HIPO leverages event-based nonlinearity to trigger self-oscillation of pneumatic actuator, which positively utilizes intrinsic characteristics of materials. This enables the system to spontaneously generate periodic control signals and directly produce motion responses, eliminating the need for incorporating external actuation components. By efficiently and rapidly converting internal energy of airflow into the kinetic energy of robots, HIPO achieves a frequency of up to 20 Hz. Furthermore, we demonstrate the versatility and high-performance capabilities of HIPO through bio-inspired robots: an insect-like fast-crawler (with speeds up to 50.27 cm/s), a high-frequency butterfly-like wing-flapper, and a maneuverable duck-like swimmer. By eliminating external components and seamlessly fusing signal generation, energy conversion, and motion output, HIPO unleashes rapid and efficient motion, unlocking potential for high-performance soft robotics.