Education
Assessing Judging Bias in Large Reasoning Models: An Empirical Study
Wang, Qian, Lou, Zhanzhi, Tang, Zhenheng, Chen, Nuo, Zhao, Xuandong, Zhang, Wenxuan, Song, Dawn, He, Bingsheng
Large Reasoning Models (LRMs) like DeepSeek-R1 and OpenAI-o1 have demonstrated remarkable reasoning capabilities, raising important questions about their biases in LLM-as-a-judge settings. We present a comprehensive benchmark comparing judging biases between LLMs and LRMs across both subjective preference-alignment datasets and objective fact-based datasets. Through investigation of bandwagon, authority, position, and distraction biases, we uncover four key findings: (1) despite their advanced reasoning capabilities, LRMs remain susceptible to the above biases; (2) LRMs demonstrate better robustness than LLMs specifically on fact-related datasets; (3) LRMs exhibit notable position bias, preferring options in later positions; and (4) we identify a novel "superficial reflection bias" where phrases mimicking reasoning (e.g., "wait, let me think...") significantly influence model judgments. To address these biases, we design and evaluate three mitigation strategies: specialized system prompts that reduce judging biases by up to 19\% in preference alignment datasets and 14\% in fact-related datasets, in-context learning that provides up to 27\% improvement on preference tasks but shows inconsistent results on factual tasks, and a self-reflection mechanism that reduces biases by up to 10\% in preference datasets and 16\% in fact-related datasets, with self-reflection proving particularly effective for LRMs. Our work provides crucial insights for developing more reliable LLM-as-a-Judge frameworks, especially as LRMs become increasingly deployed as automated judges.
CPG-EVAL: A Multi-Tiered Benchmark for Evaluating the Chinese Pedagogical Grammar Competence of Large Language Models
Purpose: The rapid emergence of large language models (LLMs) such as ChatGPT has significantly impacted foreign language education, yet their pedagogical grammar competence remains under-assessed. This paper introduces CPG-EVAL, the first dedicated benchmark specifically designed to evaluate LLMs' knowledge of pedagogical grammar within the context of foreign language instruction. Methodology: The benchmark comprises five tasks designed to assess grammar recognition, fine-grained grammatical distinction, categorical discrimination, and resistance to linguistic interference. Findings: Smaller-scale models can succeed in single language instance tasks, but struggle with multiple instance tasks and interference from confusing instances. Larger-scale models show better resistance to interference but still have significant room for accuracy improvement. The evaluation indicates the need for better instructional alignment and more rigorous benchmarks, to effectively guide the deployment of LLMs in educational contexts. Value: This study offers the first specialized, theory-driven, multi-tiered benchmark framework for systematically evaluating LLMs' pedagogical grammar competence in Chinese language teaching contexts. CPG-EVAL not only provides empirical insights for educators, policymakers, and model developers to better gauge AI's current abilities in educational settings, but also lays the groundwork for future research on improving model alignment, enhancing educational suitability, and ensuring informed decision-making concerning LLM integration in foreign language instruction.
Optimizing Multi-Gateway LoRaWAN via Cloud-Edge Collaboration and Knowledge Distillation
For large-scale multi-gateway LoRaWAN networks, this study proposes a cloud-edge collaborative resource allocation and decision-making method based on edge intelligence, HEAT-LDL (HEAT-Local Distill Lyapunov), which realizes collaborative decision-making between gateways and terminal nodes. HEAT-LDL combines the Actor-Critic architecture and the Lyapunov optimization method to achieve intelligent downlink control and gateway load balancing. When the signal quality is good, the network server uses the HEAT algorithm to schedule the terminal nodes. To improve the efficiency of autonomous decision-making of terminal nodes, HEAT-LDL performs cloud-edge knowledge distillation on the HEAT teacher model on the terminal node side. When the downlink decision instruction is lost, the terminal node uses the student model and the edge decider based on prior knowledge and local history to make collaborative autonomous decisions. Simulation experiments show that compared with the optimal results of all compared algorithms, HEAT-LDL improves the packet success rate and energy efficiency by 20.5% and 88.1%, respectively.
HEAT:History-Enhanced Dual-phase Actor-Critic Algorithm with A Shared Transformer
Although the LoRaW AN network can support a larger node scale than the LoRa private network, as the number of devices increases, the performance of the LoRaW AN network in terms of network congestion and energy consumption faces significant challenges. The limited spectrum resources and channel congestion will lead to a decrease in the communication efficiency of the netwo rk, which in turn affects the reliability of data transmission. How to achieve efficient and energy - saving resource allocation while ensuring network performance remains a key issue. In order to improve the overall performance of the LoRaW AN network, optim izing the transmission strategy parameters such as the spreading factor, transmit power, and receive window of the uplink and downlink is considered to be an effective means. By reasonably configuring these parameters, network conflicts can be effectively reduced, signal attenuation can be reduced, and signal coverage can be increased, thereby improving network reliability and communication quality. However, most of the existing optimization methods focus on the adjustment of the spreading factor and transm it power of the uplink, and rarely consider the impact of the downlink on network performance. To address this problem, this chapter proposes a History - E nhanced t wo - phase Actor - Critic a lgorithm with a s hared Transformer (HEA T), which aims to improve the resource allocation strategy of the LoRaW AN network and improve the overall performance of the network. This chapter conducts multiple sets of comparative experiments between HEA T and various popular methods under different device densities and traffic int ensities to verify the effectiveness of HEA T. 2 System Model and Problem Representation In order to efficiently verify the effectiveness of various LoRaW AN resource allocation strategies, this section describes and models the LoRa link behavior and the LoRaW AN standard in detail. Subsequently, this section proposes the target problem of LoRaW AN resource allocation and expresses the target problem as a Markov decision process.
Benchmarking Large Language Models for Calculus Problem-Solving: A Comparative Analysis
This study presents a comprehensive evaluation of five leading large language models (LLMs) - Chat GPT 4o, Copilot Pro, Gemini Advanced, Claude Pro, and Meta AI - on their performance in solving calculus differentiation problems. The investigation assessed these models across 13 fundamental problem types, employing a systematic cross-evaluation framework where each model solved problems generated by all models. Results revealed significant performance disparities, with Chat GPT 4o achieving the highest success rate (94.71%), followed by Claude Pro (85.74%), Gemini Advanced (84.42%), Copilot Pro (76.30%), and Meta AI (56.75%). All models excelled at procedural differentiation tasks but showed varying limitations with conceptual understanding and algebraic manipulation. Notably, problems involving increasing/decreasing intervals and optimization word problems proved most challenging across all models. The cross-evaluation matrix revealed that Claude Pro generated the most difficult problems, suggesting distinct capabilities between problem generation and problem-solving. These findings have significant implications for educational applications, highlighting both the potential and limitations of LLMs as calculus learning tools. While they demonstrate impressive procedural capabilities, their conceptual understanding remains limited compared to human mathematical reasoning, emphasizing the continued importance of human instruction for developing deeper mathematical comprehension.
It's not too late to stop Trump and the Silicon Valley broligarchy from controlling our lives, but we must act now Carole Cadwalladr
To walk into the lion's den once might be considered foolhardy. To do so again after being mauled by the lion? Six years ago I gave a talk at Ted, the world's leading technology and ideas conference. It led to a gruelling lawsuit and a series of consequences that reverberate through my life to this day. And last week I returned. To give another talk that would incorporate some of my experience: a Ted Talk about being sued for giving a Ted Talk, and how the lessons I'd learned from surviving all that were a model for surviving "broligarchy" – a concept I first wrote about in the Observer in July last year: the alignment of Silicon Valley and autocracy, and a kind of power the world has never seen before.
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
Fang, Luyang, Yu, Xiaowei, Cai, Jiazhang, Chen, Yongkai, Wu, Shushan, Liu, Zhengliang, Yang, Zhenyuan, Lu, Haoran, Gong, Xilin, Liu, Yufang, Ma, Terry, Ruan, Wei, Abbasi, Ali, Zhang, Jing, Wang, Tao, Latif, Ehsan, Liu, Wei, Zhang, Wei, Kolouri, Soheil, Zhai, Xiaoming, Zhu, Dajiang, Zhong, Wenxuan, Liu, Tianming, Ma, Ping
The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey provides a comprehensive analysis of two complementary paradigms: Knowledge Distillation (KD) and Dataset Distillation (DD), both aimed at compressing LLMs while preserving their advanced reasoning capabilities and linguistic diversity. We first examine key methodologies in KD, such as task-specific alignment, rationale-based training, and multi-teacher frameworks, alongside DD techniques that synthesize compact, high-impact datasets through optimization-based gradient matching, latent space regularization, and generative synthesis. Building on these foundations, we explore how integrating KD and DD can produce more effective and scalable compression strategies. Together, these approaches address persistent challenges in model scalability, architectural heterogeneity, and the preservation of emergent LLM abilities. We further highlight applications across domains such as healthcare and education, where distillation enables efficient deployment without sacrificing performance. Despite substantial progress, open challenges remain in preserving emergent reasoning and linguistic diversity, enabling efficient adaptation to continually evolving teacher models and datasets, and establishing comprehensive evaluation protocols. By synthesizing methodological innovations, theoretical foundations, and practical insights, our survey charts a path toward sustainable, resource-efficient LLMs through the tighter integration of KD and DD principles.
'Don't ask what AI can do for us, ask what it is doing to us': are ChatGPT and co harming human intelligence?
Imagine for a moment you are a child in 1941, sitting the common entrance exam for public schools with nothing but a pencil and paper. You read the following: "Write, for no more than a quarter of an hour, about a British author." Today, most of us wouldn't need 15 minutes to ponder such a question. We'd get the answer instantly by turning to AI tools such as Google Gemini, ChatGPT or Siri. Offloading cognitive effort to artificial intelligence has become second nature, but with mounting evidence that human intelligence is declining, some experts fear this impulse is driving the trend.
Russia-Ukraine war: List of key events, day 1,150
Russia launched eight missiles and 87 drones in an overnight attack on Ukraine on Saturday, causing damage in five regions across the country, the Ukrainian air force said. Air defence units shot down 33 Russian drones while another 36 were redirected by electronic warfare. Damage was recorded in five regions in the south, northeast and east. A Russian missile attack killed one person in Kharkiv, while a drone attack killed another in Sumy, with at least five children among dozens injured. Kharkiv Mayor Ihor Terekhov said 15 residential buildings, a business and an educational facility were damaged in the attack.
Unlearning Works Better Than You Think: Local Reinforcement-Based Selection of Auxiliary Objectives
Bendahi, Abderrahim, Fradin, Adrien, Lerasle, Matthieu
We introduce Local Reinforcement-Based Selection of Auxiliary Objectives (LRSAO), a novel approach that selects auxiliary objectives using reinforcement learning (RL) to support the optimization process of an evolutionary algorithm (EA) as in EA+RL framework and furthermore incorporates the ability to unlearn previously used objectives. By modifying the reward mechanism to penalize moves that do no increase the fitness value and relying on the local auxiliary objectives, LRSAO dynamically adapts its selection strategy to optimize performance according to the landscape and unlearn previous objectives when necessary. We analyze and evaluate LRSAO on the black-box complexity version of the non-monotonic Jump function, with gap parameter $\ell$, where each auxiliary objective is beneficial at specific stages of optimization. The Jump function is hard to optimize for evolutionary-based algorithms and the best-known complexity for reinforcement-based selection on Jump was $O(n^2 \log(n) / \ell)$. Our approach improves over this result to achieve a complexity of $\Theta(n^2 / \ell^2 + n \log(n))$ resulting in a significant improvement, which demonstrates the efficiency and adaptability of LRSAO, highlighting its potential to outperform traditional methods in complex optimization scenarios.