Education
Automated Visualization Makeovers with LLMs
Gangwar, Siddharth, Selby, David A., Vollmer, Sebastian J.
Making a good graphic that accurately and efficiently conveys the desired message to the audience is both an art and a science, typically not taught in the data science curriculum. Visualisation makeovers are exercises where the community exchange feedback to improve charts and data visualizations. Can multi-modal large language models (LLMs) emulate this task? Given a plot in the form of an image file, or the code used to generate it, an LLM, primed with a list of visualization best practices, is employed to semi-automatically generate constructive criticism to produce a better plot. Our system is centred around prompt engineering of a pre-trained model, relying on a combination of userspecified guidelines and any latent knowledge of data visualization practices that might lie within an LLMs training corpus. Unlike other works, the focus is not on generating valid visualization scripts from raw data or prompts, but on educating the user how to improve their existing data visualizations according to an interpretation of best practices. A quantitative evaluation is performed to measure the sensitivity of the LLM agent to various plotting issues across different chart types. We make the tool available as a simple self-hosted applet with an accessible Web interface.
CRAM: Large-scale Video Continual Learning with Bootstrapped Compression
Mall, Shivani, Henriques, Joao F.
Continual learning (CL) promises to allow neural networks to learn from continuous streams of inputs, instead of IID (independent and identically distributed) sampling, which requires random access to a full dataset. This would allow for much smaller storage requirements and self-sufficiency of deployed systems that cope with natural distribution shifts, similarly to biological learning. We focus on video CL employing a rehearsal-based approach, which reinforces past samples from a memory buffer. We posit that part of the reason why practical video CL is challenging is the high memory requirements of video, further exacerbated by long-videos and continual streams, which are at odds with the common rehearsal-buffer size constraints. To address this, we propose to use compressed vision, i.e. store video codes (embeddings) instead of raw inputs, and train a video classifier by IID sampling from this rolling buffer. Training a video compressor online (so not depending on any pre-trained networks) means that it is also subject to catastrophic forgetting. We propose a scheme to deal with this forgetting by refreshing video codes, which requires careful decompression with a previous version of the network and recompression with a new one. We name our method Continually Refreshed Amodal Memory (CRAM). We expand current video CL benchmarks to large-scale settings, namely EpicKitchens-100 and Kinetics-700, storing thousands of relatively long videos in under 2 GB, and demonstrate empirically that our video CL method outperforms prior art with a significantly reduced memory footprint.
AVA-Bench: Atomic Visual Ability Benchmark for Vision Foundation Models
Mai, Zheda, Chowdhury, Arpita, Wang, Zihe, Jeon, Sooyoung, Wang, Lemeng, Hou, Jiacheng, Chao, Wei-Lun
The rise of vision foundation models (VFMs) calls for systematic evaluation. A common approach pairs VFMs with large language models (LLMs) as general-purpose heads, followed by evaluation on broad Visual Question Answering (VQA) benchmarks. However, this protocol has two key blind spots: (i) the instruction tuning data may not align with VQA test distributions, meaning a wrong prediction can stem from such data mismatch rather than a VFM' visual shortcomings; (ii) VQA benchmarks often require multiple visual abilities, making it hard to tell whether errors stem from lacking all required abilities or just a single critical one. To address these gaps, we introduce AVA-Bench, the first benchmark that explicitly disentangles 14 Atomic Visual Abilities (AVAs) -- foundational skills like localization, depth estimation, and spatial understanding that collectively support complex visual reasoning tasks. By decoupling AVAs and matching training and test distributions within each, AVA-Bench pinpoints exactly where a VFM excels or falters. Applying AVA-Bench to leading VFMs thus reveals distinctive "ability fingerprints," turning VFM selection from educated guesswork into principled engineering. Notably, we find that a 0.5B LLM yields similar VFM rankings as a 7B LLM while cutting GPU hours by 8x, enabling more efficient evaluation. By offering a comprehensive and transparent benchmark, we hope AVA-Bench lays the foundation for the next generation of VFMs.
SuperRL: Reinforcement Learning with Supervision to Boost Language Model Reasoning
Liu, Yihao, Li, Shuocheng, Cao, Lang, Xie, Yuhang, Zhou, Mengyu, Dong, Haoyu, Ma, Xiaojun, Han, Shi, Zhang, Dongmei
Large language models are increasingly used for complex reasoning tasks where high-quality offline data such as expert-annotated solutions and distilled reasoning traces are often available. However, in environments with sparse rewards, reinforcement learning struggles to sample successful trajectories, leading to inefficient learning. At the same time, these offline trajectories that represent correct reasoning paths are not utilized by standard on-policy reinforcement learning methods. We introduce SuperRL, a unified training framework that adaptively alternates between RL and SFT. Whenever every rollout for a given instance receives zero reward, indicating the absence of a learning signal, SuperRL falls back to SFT on the curated offline data. Extensive experiments across diverse reasoning benchmarks show that SuperRL surpasses vanilla RL by delivering higher sample efficiency, stronger generalization, and improved robustness under sparse rewards.
CUB: Benchmarking Context Utilisation Techniques for Language Models
Hagstrรถm, Lovisa, Kim, Youna, Yu, Haeun, Lee, Sang-goo, Johansson, Richard, Cho, Hyunsoo, Augenstein, Isabelle
Incorporating external knowledge is crucial for knowledge-intensive tasks, such as question answering and fact checking. However, language models (LMs) may ignore relevant information that contradicts outdated parametric memory or be distracted by irrelevant contexts. While many context utilisation manipulation techniques (CMTs) have recently been proposed to alleviate these issues, few have seen systematic comparison. In this paper, we develop CUB (Context Utilisation Benchmark) - the first comprehensive benchmark designed to help practitioners within retrieval-augmented generation (RAG) diagnose CMTs under different context conditions. With this benchmark, we conduct the most extensive evaluation to date of seven state-of-the-art methods, representative of the main categories of CMTs, across three diverse datasets and tasks, applied to nine LMs. Our results reveal that most existing CMTs struggle to handle the full spectrum of context types encountered in real-world retrieval-augmented scenarios. We also find that many CMTs display inflated performance on simple synthesised datasets, compared to more realistic datasets with naturally occurring samples. Our findings expose critical gaps in current CMT evaluation practices and demonstrate the need for holistic testing and the development of CMTs that can robustly handle multiple context types.
Single-Pass Document Scanning for Question Answering
Cao, Weili, Wang, Jianyou, Zheng, Youze, Bao, Longtian, Zheng, Qirui, Berg-Kirkpatrick, Taylor, Paturi, Ramamohan, Bergen, Leon
Handling extremely large documents for question answering is challenging: chunk-based embedding methods often lose track of important global context, while full-context transformers can be prohibitively expensive for hundreds of thousands of tokens. We propose a single-pass document scanning approach that processes the entire text in linear time, preserving global coherence while deciding which sentences are most relevant to the query. On 41 QA benchmarks, our single-pass scanner consistently outperforms chunk-based embedding methods and competes with large language models at a fraction of the computational cost. By conditioning on the entire preceding context without chunk breaks, the method preserves global coherence, which is especially important for long documents. Overall, single-pass document scanning offers a simple solution for question answering over massive text. All code, datasets, and model checkpoints are available at https://github.com/MambaRetriever/MambaRetriever
OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
Ahmad, Wasi Uddin, Narenthiran, Sean, Majumdar, Somshubra, Ficek, Aleksander, Jain, Siddhartha, Huang, Jocelyn, Noroozi, Vahid, Ginsburg, Boris
Since the advent of reasoning-based large language models, many have found great success from distilling reasoning capabilities into student models. Such techniques have significantly bridged the gap between reasoning and standard LLMs on coding tasks. Despite this, much of the progress on distilling reasoning models remains locked behind proprietary datasets or lacks details on data curation, filtering and subsequent training. To address this, we construct a superior supervised fine-tuning (SFT) dataset that we use to achieve state-of-the-art coding capability results in models of various sizes. Our distilled models use only SFT to achieve 61.8% on LiveCodeBench and 24.6% on CodeContests, surpassing alternatives trained with reinforcement learning. We then perform analysis on the data sources used to construct our dataset, the impact of code execution filtering, and the importance of instruction/solution diversity. We observe that execution filtering negatively affected benchmark accuracy, leading us to prioritize instruction diversity over solution correctness. Finally, we also analyze the token efficiency and reasoning patterns utilized by these models. We will open-source these datasets and distilled models to the community.
Digital resurrection: fascination and fear over the rise of the deathbot
Rod Stewart had a few surprise guests at a recent concert in Charlotte, North Carolina. His old friend Ozzy Osbourne, the lead singer of Black Sabbath who died last month, was apparently beamed in from some kind of rock heaven, where he was reunited with other departed stars including Michael Jackson, Tina Turner and Bob Marley. The AI-generated images divided Stewart's fans. Some denounced them as disrespectful and distasteful; others found the tribute beautiful. At about the same time, another AI controversy erupted when Jim Acosta, a former CNN White House correspondent, interviewed a digital recreation of Joaquin Oliver, who was killed at the age of 17 in a 2018 high school shooting in Florida.
Texas company creates drones to confront school shooters in seconds
Campus Guardian Angel founder and CEO Justin Marston tells'Fox & Friends First' about his Texas-based company's drones designed to make schools safer by confronting an active shooter within seconds. A drone system designed to confront a school shooter within seconds could soon become a frontline defense in classrooms across America. Texas-based Campus Guardian Angel has developed the technology which stations drones inside schools, ready to deploy the moment an emergency alert is triggered. The drones, all controlled remotely at a central operation center in Austin, Texas, are stored in charging boxes inside schools. Once activated, they are designed to fire powder pellets to incapacitate a shooter within 60 seconds and buy time for local law enforcement to arrive at the scene.