Goto

Collaborating Authors

 Education


A Comparative Study of Competency Question Elicitation Methods from Ontology Requirements

arXiv.org Artificial Intelligence

Competency Questions (CQs) are pivotal in knowledge engineering, guiding the design, validation, and testing of ontologies. A number of diverse formulation approaches have been proposed in the literature, ranging from completely manual to Large Language Model (LLM) driven ones. However, attempts to characterise the outputs of these approaches and their systematic comparison are scarce. This paper presents an empirical comparative evaluation of three distinct CQ formulation approaches: manual formulation by ontology engineers, instantiation of CQ patterns, and generation using state of the art LLMs. We generate CQs using each approach from a set of requirements for cultural heritage, and assess them across different dimensions: degree of acceptability, ambiguity, relevance, readability and complexity. Our contribution is twofold: (i) the first multi-annotator dataset of CQs generated from the same source using different methods; and (ii) a systematic comparison of the characteristics of the CQs resulting from each approach. Our study shows that different CQ generation approaches have different characteristics and that LLMs can be used as a way to initially elicit CQs, however these are sensitive to the model used to generate CQs and they generally require a further refinement step before they can be used to model requirements.


We Need Knowledge Distillation for Solving Math Word Problems

arXiv.org Artificial Intelligence

The enhancement of mathematical capabilities in large language models (LLMs) fosters new developments in mathematics education within primary and secondary schools, particularly as they relate to intelligent tutoring systems. However, LLMs require substantial computational resources, resulting in significant costs in educational contexts. To mitigate this drawback, this paper investigates the feasibility of compressing LLMs for solving math word problems (MWPs). We compress the embedded vectors encoded by BERT and distill a considerably smaller student model. Our findings indicate that the student model can maintain nearly 90% of the performance of the teacher model while utilizing only 1/12 of its parameters. In addition to achieving high accuracy, the model exhibits strong generalizability, as the compressed vectors perform well across all tasks related to MWPs, and the distillation process is not task-specific. The success of this distillation demonstrates that the underlying principles are generic and not limited to a specific task. We further explore the reasons behind the compressibility of embedded vectors, revealing that part-of-speech information, rather than entity recognition, is crucial for MWPs, which may significantly contribute to their compressibility. The improvements in efficiency and cost reduction provide substantial value for intelligent tutoring systems and significantly advance the field of intelligent education.


A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks

arXiv.org Artificial Intelligence

Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and are often easy to perform manually, posing a significant threat to the safe and secure deployment of LLM-based systems. We study the effectiveness of the Crescendo multi-turn jailbreak at the level of intermediate model representations and find that safety-aligned LMs often represent Crescendo responses as more benign than harmful, especially as the number of conversation turns increases. Our analysis indicates that at each turn, Crescendo prompts tend to keep model outputs in a "benign" region of representation space, effectively tricking the model into fulfilling harmful requests. Further, our results help explain why single-turn jailbreak defenses like circuit breakers are generally ineffective against multi-turn attacks, motivating the development of mitigations that address this generalization gap.


Toward Cyclic A.I. Modelling of Self-Regulated Learning: A Case Study with E-Learning Trace Data

arXiv.org Artificial Intelligence

Many e-learning platforms assert their ability or potential to improve students' self-regulated learning (SRL), however the cyclical and undirected nature of SRL theoretical models represent significant challenges for representation within contemporary machine learning frameworks. We apply SRL-informed features to trace data in order to advance modelling of students' SRL activities, to improve predictability and explainability regarding the causal effects of learning in an eLearning environment. We demonstrate that these features improve predictive accuracy and validate the value of further research into cyclic modelling techniques for SRL.


Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models

arXiv.org Artificial Intelligence

Deploying robots in human-centric settings like households requires balancing robot autonomy with humans' sense of agency [1, 2, 3, 4, 5, 6]. Full teleoperation offers users fine-grained control but imposes a high cognitive load, whereas fully autonomous robots act independently but often misalign their actions with nuanced human needs. Assistive teleoperation -- a paradigm in which both the human and the robot share control [7, 8, 9, 10] -- has thus emerged as an ideal middle ground. By keeping the user in control of high-level decisions while delegating low-level actions to the autonomous robot, this approach both preserves user agency and enhances overall system performance. As such, assistive teleoperation is becoming a desirable paradigm for robots to serve as reliable partners in human-centric environments, such as assisting individuals with motor impairments [11, 12]. While promising, assistive teleoperation in everyday environments remains challenging. A longstanding challenge in assistive teleoperation is to infer human intents from user control inputs and assist users with correct actions [8]. This challenge is amplified in real-world settings, where robots must go beyond closed-set intent prediction [13, 14] to handle diverse, open-ended user goals across different contexts and scenes. As a result, a key capability the robot should possess is to interpret user control inputs within the visual context and infer intent through commonsense reasoning.


This teen 3D printed a beehive for his bedroom

Popular Science

Breakthroughs, discoveries, and DIY tips sent every weekday. While many 13-year-old boys might spend their summers playing video games or attending camp, Oliver Taylor decided to build a custom-made, 3D-printed beehive--in his bedroom. Oliver, who lives in Utah, built the DIY insect habitat with two hexagonal, 3D-printed units connected to his bedroom window. Bees enter through a ventilation tube attached to the window, which slightly resembles a stand-up air conditioning unit. The hexagonal hives are modular in design, meaning Oliver can theoretically continue expanding their size by connecting additional units.


The AI Birthday Letter That Blew Me Away

The Atlantic - Technology

In May, I asked Google's chatbot, Gemini, to write a birthday letter to my best friend. Within seconds, it spat out the most impressive piece of AI writing I have ever encountered. Instead of reading as soulless, machine-generated text, the letter felt unnervingly like something I might've actually written. "You're probably rolling your eyes," the letter read, after a sentence that my friend would most definitely have rolled his eyes at. All I had typed into the chatbot was a nine-word prompt containing my friend's first name and the age he was turning.


Schools turn to handwritten exams as AI cheating surges

FOX News

A growing number of fire departments across the country are turning to artificial intelligence to help detect and respond to wildfires more quickly. The rise of artificial intelligence in education is forcing schools and universities to rethink everything from homework policies to how final exams are administered. With tools like ChatGPT now widespread, students can generate essays, solve complex math problems or draft lab reports in seconds, raising urgent questions about what authentic learning looks like in 2025. To fight back, some schools are turning to an unlikely solution: pen and paper. The old-school "blue book," a lined booklet used for handwritten test answers, is staging a comeback, according to reporting from The Wall Street Journal.


Enhancing Clinical Multiple-Choice Questions Benchmarks with Knowledge Graph Guided Distractor Generation

arXiv.org Artificial Intelligence

Clinical tasks such as diagnosis and treatment require strong decision-making abilities, highlighting the importance of rigorous evaluation benchmarks to assess the reliability of large language models (LLMs). In this work, we introduce a knowledge-guided data augmentation framework that enhances the difficulty of clinical multiple-choice question (MCQ) datasets by generating distractors (i.e., incorrect choices that are similar to the correct one and may confuse existing LLMs). Using our KG-based pipeline, the generated choices are both clinically plausible and deliberately misleading. Our approach involves multi-step, semantically informed walks on a medical knowledge graph to identify distractor paths-associations that are medically relevant but factually incorrect-which then guide the LLM in crafting more deceptive distractors. We apply the designed knowledge graph guided distractor generation (KGGDG) pipline, to six widely used medical QA benchmarks and show that it consistently reduces the accuracy of state-of-the-art LLMs. These findings establish KGGDG as a powerful tool for enabling more robust and diagnostic evaluations of medical LLMs.


DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment

arXiv.org Artificial Intelligence

--We introduce DeST A2.5-Audio, a general-purpose Large Audio Language Model (LALM) designed for robust auditory perception and instruction-following, without requiring task-specific audio instruction-tuning. Recent LALMs typically augment Large Language Models (LLMs) with auditory capabilities by training on large-scale, manually curated or LLM-synthesized audio-instruction datasets. However, these approaches have often suffered from the catastrophic forgetting of the LLM's original language abilities. T o address this, we revisit the data construction pipeline and propose DeST A, a self-generated cross-modal alignment strategy in which the backbone LLM generates its own training targets. This approach preserves the LLM's native language proficiency while establishing effective audio-text alignment, thereby enabling zero-shot generalization without task-specific tuning. Using DeST A, we construct DeST A-AQA5M, a large-scale, task-agnostic dataset containing 5 million training samples derived from 7,000 hours of audio spanning 50 diverse datasets, including speech, environmental sounds, and music. DeST A2.5-Audio achieves state-of-the-art or competitive performance across a wide range of audio-language benchmarks, including Dynamic-SUPERB, MMAU, SAKURA, Speech-IFEval, and V oiceBench. Comprehensive comparative studies demonstrate that our self-generated strategy outperforms widely adopted data construction and training strategies in both auditory perception and instruction-following capabilities. Our findings underscore the importance of carefully designed data construction in LALM development and offer practical insights for building robust, general-purpose LALMs. HE development of general-purpose artificial intelligence has become a central focus in contemporary AI research, driven by the remarkable performance of large language models (LLMs) across various natural language understanding and generation tasks [1]-[7]. Building on these advancements, a promising direction is to equip LLMs with multi-modal understanding capabilities, leading to the emergence of Large Audio Language Models (LALMs) [8]-[22] and Large Vision Language Models (L VLMs) [23]-[27]. This paper focuses on building a general-purpose LALM, illustrated in Figure 1. To develop a general-purpose LALM, two core capabilities are essential: auditory perception and instruction-following. Auditory perception refers to the comprehensive processing of auditory information, including speech, non-verbal cues, background sounds, and music.