sequential review mechanism
LOCA-R: Near-Perfect Performance on the Chinese Physics Olympiad 2025
Jian, Dong-Shan, Li, Xiang, Yan, Chen-Xu, Zheng, Hui-Wen, Bian, Zhi-Zhang, Fang, You-Le, Zhang, Sheng-Qi, Gong, Bing-Rui, He, Ren-Xi, Zhang, Jing-Tian, Meng, Ce, Ma, Yan-Qing
Olympiad-level physics problem-solving presents a significant challenge for both humans and artificial intelligence (AI), as it requires a sophisticated integration of precise calculation, abstract reasoning, and a fundamental grasp of physical principles. The Chinese Physics Olympiad (CPhO), renowned for its complexity and depth, serves as an ideal and rigorous testbed for these advanced capabilities. In this paper, we introduce LOCA-R (LOgical Chain Augmentation for Reasoning), an improved version of the LOCA framework adapted for complex reasoning, and apply it to the CPhO 2025 theory examination. LOCA-R achieves a near-perfect score of 313 out of 320 points, solidly surpassing the highest-scoring human competitor and significantly outperforming all baseline methods. Physics problem-solving, or more generally complex scientific reasoning, stands as a challenging frontier for both humans and AI. It demands not only mathematical derivations, but also the ability to translate complex real-world scenarios described in natural language into abstract models. This process requires a deep understanding of physical laws to select and apply the appropriate principles. Therefore, given the strict demand for accuracy in scientific problems, the most critical task is to maximize the problem-solving capabilities of LLMs, pushing them towards perfect scores. The trade-off between performance and computational cost, while important, is a subsequent consideration.
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.95)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.95)
Eliciting Honest Information From Authors Using Sequential Review
Zhang, Yichi, Schoenebeck, Grant, Su, Weijie
In the setting of conference peer review, the conference aims to accept high-quality papers and reject low-quality papers based on noisy review scores. A recent work proposes the isotonic mechanism, which can elicit the ranking of paper qualities from an author with multiple submissions to help improve the conference's decisions. However, the isotonic mechanism relies on the assumption that the author's utility is both an increasing and a convex function with respect to the review score, which is often violated in peer review settings (e.g.~when authors aim to maximize the number of accepted papers). In this paper, we propose a sequential review mechanism that can truthfully elicit the ranking information from authors while only assuming the agent's utility is increasing with respect to the true quality of her accepted papers. The key idea is to review the papers of an author in a sequence based on the provided ranking and conditioning the review of the next paper on the review scores of the previous papers. Advantages of the sequential review mechanism include 1) eliciting truthful ranking information in a more realistic setting than prior work; 2) improving the quality of accepted papers, reducing the reviewing workload and increasing the average quality of papers being reviewed; 3) incentivizing authors to write fewer papers of higher quality.
- North America > United States > Michigan (0.04)
- North America > United States > Pennsylvania (0.04)
- North America > United States > New York > New York County > New York City (0.04)