Large Language Model
Data Therapist: Eliciting Domain Knowledge from Subject Matter Experts Using Large Language Models
Shin, Sungbok, Jeon, Hyeon, Hong, Sanghyun, Elmqvist, Niklas
Effective data visualization requires not only technical proficiency but also a deep understanding of the domain-specific context in which data exists. This context often includes tacit knowledge about data provenance, quality, and intended use, which is rarely explicit in the dataset itself. Motivated by growing demands to surface tacit knowledge, we present the Data Therapist, a web-based system that helps domain experts externalize such implicit knowledge through a mixed-initiative process combining iterative Q&A with interactive annotation. Powered by a large language model, the system automatically analyzes user-supplied datasets, prompts users with targeted questions, and supports annotation at varying levels of granularity. The resulting structured knowledge base can inform both human and automated visualization design. A qualitative study with expert pairs from Accounting, Political Science, and Computer Security revealed recurring patterns in how expert reason about their data and highlighted opportunities for AI support to enhance visualization design.
W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
The demand for efficient natural language processing (NLP) systems has led to the development of lightweight language models. Previous work in this area has primarily focused on manual design or training-based neural architecture search (NAS) methods. Recently, zero-shot NAS methods have been proposed for evaluating language models without the need for training. However, prevailing approaches to zero-shot NAS often face challenges such as biased evaluation metrics and computational inefficiencies. In this paper, we introduce weight-weighted PCA (W-PCA), a novel zero-shot NAS method specifically tailored for lightweight language models. Our approach utilizes two evaluation proxies: the parameter count and the number of principal components with cumulative contribution exceeding $η$ in the feed-forward neural (FFN) layer. Additionally, by eliminating the need for gradient computations, we optimize the evaluation time, thus enhancing the efficiency of designing and evaluating lightweight language models. We conduct a comparative analysis on the GLUE and SQuAD datasets to evaluate our approach. The results demonstrate that our method significantly reduces training time compared to one-shot NAS methods and achieves higher scores in the testing phase compared to previous state-of-the-art training-based methods. Furthermore, we perform ranking evaluations on a dataset sampled from the FlexiBERT search space. Our approach exhibits superior ranking correlation and further reduces solving time compared to other zero-shot NAS methods that require gradient computation.
Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
Taghibakhshi, Ali, Sreenivas, Sharath Turuvekere, Muralidharan, Saurav, Chochowski, Marcin, Karnati, Yashaswi, Joshi, Raviraj, Mahabaleshwarkar, Ameya Sunil, Chen, Zijia, Suhara, Yoshi, Olabiyi, Oluwatobi, Korzekwa, Daniel, Patwary, Mostofa, Shoeybi, Mohammad, Kautz, Jan, Catanzaro, Bryan, Aithal, Ashwath, Tajbakhsh, Nima, Molchanov, Pavlo
Hybrid LLM architectures that combine Attention and State Space Models (SSMs) achieve state-of-the-art accuracy and runtime performance. Recent work has demonstrated that applying compression and distillation to Attention-only models yields smaller, more accurate models at a fraction of the training cost. In this work, we explore the effectiveness of compressing Hybrid architectures. We introduce a novel group-aware pruning strategy that preserves the structural integrity of SSM blocks and their sequence modeling capabilities. Furthermore, we demonstrate the necessity of such SSM pruning to achieve improved accuracy and inference speed compared to traditional approaches. Our compression recipe combines SSM, FFN, embedding dimension, and layer pruning, followed by knowledge distillation-based retraining, similar to the MINITRON technique. Using this approach, we compress the Nemotron-H 8B Hybrid model down to 4B parameters with up to 40x fewer training tokens. The resulting model surpasses the accuracy of similarly-sized models while achieving 2x faster inference, significantly advancing the Pareto frontier.
HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving
Kumar, Avinash, Nag, Shashank, Clemons, Jason, John, Lizy, Das, Poulami
Early-Exit Large Language Models (EE-LLMs) enable high throughput inference by allowing tokens to exit early at intermediate layers. However, their throughput is limited by the computational and memory savings. Existing EE-LLM frameworks rely on a single model and therefore, their token generation latencies are bottlenecked by tokens that do not exit early and traverse additional layers. Moreover, early exits are only known at runtime and depend on the request. Therefore, these frameworks load the weights of all model layers even though large portions remain unused when tokens exit early. The lack of memory savings limit us from scaling the batch sizes. We propose $\textit{HELIOS}$, a framework that improves both token generation latency and batch sizes to enable high-throughput in EE-LLMs. HELIOS exploits two insights. $\textit{First}$, early exits are often complimentary across models, tokens that do not exit early on one model often take an early-exit on another. HELIOS employs multiple models and dynamically switches between them to collectively maximize the number of tokens that exit early, and minimize token generation latencies. $\textit{Second}$, even when a predicted token does not exit early due to poor confidence, it often remains unchanged even after additional layer traversal. HELIOS greedily allows such tokens to exit early and only loads the weights of the most likely to be used layers, yielding memory savings which is then re-purposed to increase batch sizes. HELIOS employs real-time profiling to accurately identify the early-exit distributions, and adaptively switches between models by tracking tokens in real-time to minimize the performance degradation caused by greedy model loading and exiting. Our evaluations show that HELIOS achieves $1.48\times$ higher throughput and $15.14\times$ larger batch size compared to existing EE-LLM frameworks.
TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
Moroshan, Vladyslav, Siems, Julien, Zela, Arber, Carstensen, Timur, Hutter, Frank
This paper presents TempoPFN, a univariate time series foundation model based on linear Recurrent Neural Networks (RNNs) pre-trained exclusively on synthetic data. The model uses a GatedDeltaProduct architecture with state-weaving for fully parallelizable training across sequence lengths, eliminating the need for windowing or summarization techniques while maintaining robust temporal state-tracking. Our comprehensive synthetic data pipeline unifies diverse generators--including stochastic differential equations, Gaussian processes, and audio synthesis--with novel augmentations. In zero-shot evaluations on the Gift-Eval benchmark, TempoPFN achieves top-tier competitive performance, outperforming all existing synthetic-only approaches and surpassing the majority of models trained on real-world data, while being more efficient than existing baselines by leveraging fully paralleliz-able training and inference. Recent advances in large language models have inspired foundation models for time series forecasting that enable zero-shot predictions across diverse datasets without fine-tuning (Ansari et al., 2024; Das et al., 2024; Woo et al., 2024; Auer et al., 2025). By treating historical observations as input context, these models democratize forecasting for non-experts and excel in data-scarce domains. However, current approaches face critical limitations. While non-linear RNNs like those in TiReX (Auer et al., 2025) maintain temporal state, they require sequential processing that limits scalability. Although some recent models attempt synthetic-only pre-training including ForecastPFN (Dooley et al., 2023), CauKer (Xie et al., 2024), and Mamba4Cast (Bhethanabhotla & Swelam, 2024) none reported state-of-the-art performance on the Gift-Eval benchmark. TabPFN-TS (Hoo et al., 2024), which adapts a tabular foundation model to time series, achieves strong Gift-Eval performance but does not release its synthetic pre-training data, limiting reproducibility and extensibility. Figure 1: (Left) Synthetic Data Generation pipeline containing a mix of novel and existing time-series generators are augmented with a diverse set of augmentations to produce the time-series used for training. We introduce T empoPFN (see Table 1 and Figure 1), a time series forecasting foundation model using linear RNNs with GatedDeltaProduct recurrence (Siems et al., 2025) for parallelizable training and inference across the sequence length. Unlike TiRex (Auer et al., 2025) which argued that non-linear RNNs like sLSTM are necessary for time-series forecasting due to their state-tracking capabilities we find that linear RNNs based on the GatedDeltaProduct recurrence are sufficient, in line with recent research demonstrating how linear RNNs can perform state-tracking (Grazzi et al., 2025).
Causal Masking on Spatial Data: An Information-Theoretic Case for Learning Spatial Datasets with Unimodal Language Models
Junkin, Jared, Nathanson, Samuel
Language models are traditionally designed around causal masking. In domains with spatial or relational structure, causal masking is often viewed as inappropriate, and sequential linearizations are instead used. Yet the question of whether it is viable to accept the information loss introduced by causal masking on nonsequential data has received little direct study, in part because few domains offer both spatial and sequential representations of the same dataset. In this work, we investigate this issue in the domain of chess, which naturally supports both representations. We train language models with bidirectional and causal self-attention mechanisms on both spatial (board-based) and sequential (move-based) data. Our results show that models trained on spatial board states - \textit{even with causal masking} - consistently achieve stronger playing strength than models trained on sequential data. While our experiments are conducted on chess, our results are methodological and may have broader implications: applying causal masking to spatial data is a viable procedure for training unimodal LLMs on spatial data, and in some domains is even preferable to sequentialization.
Has OpenAI really made ChatGPT better for users with mental health problems?
ChatGPT on App Store displayed on a phone screen on 07 June 2025. ChatGPT on App Store displayed on a phone screen on 07 June 2025. Has OpenAI really made ChatGPT better for users with mental health problems? Prompts indicating suicidal ideation got alarming replies, which experts say shows'how easy it is to break the model' A n OpenAI statement released this week claimed the company had made its popular service ChatGPT better at supporting users experiencing mental health problems like suicidal ideation or delusions, but experts tell the Guardian they need to do more to truly ensure users are protected. The Guardian tested several prompts indicating suicidal ideation with the ChatGPT GPT-5 updated model, which is now the default, and got alarming responses from the large language model (LLM) chatbot.
'A lot of this is speculative': faith and fear mix amid 3tn global datacentre boom
Several new sites such as this are in the pipeline in the UK. Several new sites such as this are in the pipeline in the UK. 'A lot of this is speculative': faith and fear mix amid $3tn global datacentre boom The global investment spree in artificial intelligence is producing some remarkable numbers and a projected $3tn (£2.3tn) spend on datacentres is one of them. These vast warehouses are the central nervous system of AI tools such as OpenAI's ChatGPT and Google's Veo 3, underpinning the training and operation of a technology into which investors have poured vast sums of money. Despite concerns that the AI boom could be a bubble waiting to burst, there are few signs of it at the moment.
Are YOU addicted to ChatGPT? Scientists warn something strange is happening to people who use AI too often
How Andrew's'rude' comment about Kate sparked bitter feud between ex-prince and William - who'couldn't wait for the day' when Charles finally threw him out Once a typical Californian'blue' enclave, a beachside paradise is now burning red... and it's coming for Gavin Newsom All the winning cards are now in her hands. I know her next move - it's devastating'I saw Aileen Wournos 12 hours before she was executed and she finally admitted she was a serial killer': How the 46-year-old executed for murdering seven men in just one year confessed her sins to her best friend in their final meeting Nancy Mace accused of throwing explosive airport tantrum at cops after curb pickup mix-up... as she fires back See the best celebrity costumes from Heidi Klum's iconic 2025 Halloween party... and the scariest Watch'naked nanny' accused of murdering hero grandpa with screwdriver as she frolics with 2-year-old in new videos... and her dark spiral is revealed Outrage over America's worst school where students fight, smoke weed and have sex in full view of horrified neighbors The whispers about Oprah's best girl Gayle King are reaching fever pitch among all my media friends. ISIS-inspired terror plot hatched by'homegrown radicals' thwarted by FBI as agents raid suburban home and arrest child New York City Marathon legend Dave Obelkevich, 82, reveals what's kept him pounding NYC streets for five decades We lost 100 lbs without taking'easy way out' Ozempic by using these'traditional' methods: They're simple daily habits... that ended our 1,000-calorie donuts binges for good Are YOU addicted to ChatGPT? People who use AI too often are experiencing a strange and concerning new psychological condition, experts have warned. Psychologists say that fans of popular chatbots like ChatGPT, Claude, and Replika are at risk of becoming addicted to AI.
The Ray Tracing Sampler: Bayesian Sampling of Neural Networks for Everyone
We derive a Markov Chain Monte Carlo sampler based on following ray paths in a medium where the refractive index $n(x)$ is a function of the desired likelihood $\mathcal{L}(x)$. The sampling method propagates rays at constant speed through parameter space, leading to orders of magnitude higher resilience to heating for stochastic gradients as compared to Hamiltonian Monte Carlo (HMC), as well as the ability to cross any likelihood barrier, including holes in parameter space. Using the ray tracing method, we sample the posterior distributions of neural network outputs for a variety of different architectures, up to the 1.5 billion-parameter GPT-2 (Generative Pre-trained Transformer 2) architecture, all on a single consumer-level GPU. We also show that prior samplers including traditional HMC, microcanonical HMC, Metropolis, Gibbs, and even Monte Carlo integration are special cases within a generalized ray tracing framework, which can sample according to an arbitrary weighting function. Public code and documentation for C, JAX, and PyTorch are available at https://bitbucket.org/pbehroozi/ray-tracing-sampler/src