Large Language Model
Comparing representations of long clinical texts for the task of patient note-identification
Alsaidi, Safa, Vincent, Marc, Boyer, Olivia, Garcelon, Nicolas, Couceiro, Miguel, Coulet, Adrien
In this paper, we address the challenge of patient-note identification, which involves accurately matching an anonymized clinical note to its corresponding patient, represented by a set of related notes. This task has broad applications, including duplicate records detection and patient similarity analysis, which require robust patient-level representations. We explore various embedding methods, including Hierarchical Attention Networks (HAN), three-level Hierarchical Transformer Networks (HTN), LongFormer, and advanced BERT-based models, focusing on their ability to process mediumto-long clinical texts effectively. Additionally, we evaluate different pooling strategies (mean, max, and mean_max) for aggregating wordlevel embeddings into patient-level representations and we examine the impact of sliding windows on model performance. Our results indicate that BERT-based embeddings outperform traditional and hierarchical models, particularly in processing lengthy clinical notes and capturing nuanced patient representations. Among the pooling strategies, mean_max pooling consistently yields the best results, highlighting its ability to capture critical features from clinical notes. Furthermore, the reproduction of our results on both MIMIC dataset and Necker hospital data warehouse illustrates the generalizability of these approaches to real-world applications, emphasizing the importance of both embedding methods and aggregation strategies in optimizing patient-note identification and enhancing patient-level modeling.
Google's new experimental AI model, Gemini 2.5 Pro, is now available to free users too
Non-paying Gemini users can now play around with Google's newest model, the experimental version of Gemini 2.5 Pro. The company announced this weekend that it's making Gemini 2.5 Pro (experimental) free for everyone to use, albeit with tighter rate limits for non-subscribers. Google introduced Gemini 2.5 Pro just last week, touting it as its "most intelligent AI model" yet, and rolled it out to Gemini Advanced users first. While free users can now try it out too, Google added that "Gemini Advanced users have expanded access and a significantly larger context window." Gemini 2.5 Pro (experimental) is the first of Google's Gemini 2.5 "thinking" models, which are said to deliver more accurate results through reasoning.
This is what happens when you fuse ChatGPT, Gemini, Midjourney, and Meta AI
There's nothing quite like when you finally get that perfect ChatGPT prompt and the little robot starts blasting through your work, only to get the notification that you're out of GPT-4 credits. You could pay for a premium account, but if you do that for every AI tool you rely on, then it starts getting really expensive. The alternative is to switch to a platform that gives you lifetime access to the same AI models you're used to, minus the monthly fee. Meta AI, Gemini, Midjourney, and more all under one roof, and a lifetime subscription just went on sale for 29.97. Not only do you get GPT-4 and GPT-4 Turbo for advanced content tasks, but you also get access to Claude 3 from Anthropic, including the Opus, Sonnet, and Haiku versions, which are great for nuanced and creative writing.
How and why parents and teachers are introducing young children to AI
Since the release of ChatGPT in late 2022, generative artificial intelligence has trickled down from adults in their offices to university students in campus libraries to teenagers in high school hallways. Now it's reaching the youngest among us, and parents and teachers are grappling with the most responsible way to introduce their under-13s to a new technology that may fundamentally reshape the future. Though the terms of service for ChatGPT, Google's Gemini and other AI models specify that the tools are only meant for those over 13, parents and teachers are taking the matter of AI education into their own hands. Inspired by a story we published on parents who are teaching their children to use AI to set them up for success in school and at work, we asked Guardian readers how and why โ or why not โ others are doing the same. Though our original story only concerned parents, we have also included teachers in the responses published below, as preparing children for future studies and jobs is one of educators' responsibilities as well.
Carbon Footprint Evaluation of Code Generation through LLM as a Service
Vartziotis, Tina, Schmidt, Maximilian, Dasoulas, George, Dellatolas, Ippolyti, Attademo, Stefano, Le, Viet Dung, Wiechmann, Anke, Hoffmann, Tim, Keckeisen, Michael, Kotsopoulos, Sotirios
Due to increased computing use, data centers consume and emit a lot of energy and carbon. These contributions are expected to rise as big data analytics, digitization, and large AI models grow and become major components of daily working routines. To reduce the environmental impact of software development, green (sustainable) coding and claims that AI models can improve energy efficiency have grown in popularity. Furthermore, in the automotive industry, where software increasingly governs vehicle performance, safety, and user experience, the principles of green coding and AI-driven efficiency could significantly contribute to reducing the sector's environmental footprint. We present an overview of green coding and metrics to measure AI model sustainability awareness. This study introduces LLM as a service and uses a generative commercial AI language model, GitHub Copilot, to auto-generate code. Using sustainability metrics to quantify these AI models' sustainability awareness, we define the code's embodied and operational carbon.
Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base
Song, Linxin, Ding, Xuwei, Zhang, Jieyu, Shi, Taiwei, Shimizu, Ryotaro, Gupta, Rahul, Liu, Yang, Kang, Jian, Zhao, Jieyu
Large language models (LLMs) possess impressive linguistic capabilities but often fail to faithfully retain factual knowledge, leading to hallucinations and unreliable outputs. Understanding LLMs' knowledge deficiencies by exhaustively evaluating against full-scale knowledge bases is computationally prohibitive, especially for closed-weight models. We propose stochastic error ascent (SEA), a scalable and efficient framework for discovering knowledge deficiencies (errors) in closed-weight LLMs under a strict query budget. Rather than naively probing all knowledge candidates, SEA formulates error discovery as a stochastic optimization process: it iteratively retrieves new high-error candidates by leveraging the semantic similarity to previously observed failures. To further enhance search efficiency and coverage, SEA employs hierarchical retrieval across document and paragraph levels, and constructs a relation directed acyclic graph to model error propagation and identify systematic failure modes. Empirically, SEA uncovers 40.7x more knowledge errors than Automated Capability Discovery and 26.7% more than AutoBencher, while reducing the cost-per-error by 599x and 9x, respectively. Human evaluation confirms the high quality of generated questions, while ablation and convergence analyses validate the contribution of each component in SEA. Further analysis on the discovered errors reveals correlated failure patterns across LLM families and recurring deficits, highlighting the need for better data coverage and targeted fine-tuning in future LLM development.
Speculative End-Turn Detector for Efficient Speech Chatbot Assistant
Ok, Hyunjong, Yoo, Suho, Lee, Jaeho
Spoken dialogue systems powered by large language models have demonstrated remarkable abilities in understanding human speech and generating appropriate spoken responses. However, these systems struggle with end-turn detection (ETD) -- the ability to distinguish between user turn completion and hesitation. This limitation often leads to premature or delayed responses, disrupting the flow of spoken conversations. In this paper, we introduce the ETD Dataset, the first public dataset for end-turn detection. The ETD dataset consists of both synthetic speech data generated with text-to-speech models and real-world speech data collected from web sources. We also propose SpeculativeETD, a novel collaborative inference framework that balances efficiency and accuracy to improve real-time ETD in resource-constrained environments. Our approach jointly employs a lightweight GRU-based model, which rapidly detects the non-speaking units in real-time on local devices, and a high-performance Wav2vec-based model running on the server to make a more challenging classification of distinguishing turn ends from mere pauses. Experiments demonstrate that the proposed SpeculativeETD significantly improves ETD accuracy while keeping the required computations low. Datasets and code will be available after the review.
Agent-Centric Personalized Multiple Clustering with Multi-Modal LLMs
Chen, Ziye, Duan, Yiqun, Zhu, Riheng, Sun, Zhenbang, Gong, Mingming
Personalized multiple clustering aims to generate diverse partitions of a dataset based on different user-specific aspects, rather than a single clustering. It has recently drawn research interest for accommodating varying user preferences. Recent approaches primarily use CLIP embeddings with proxy learning to extract representations biased toward user clustering preferences. However, CLIP primarily focuses on coarse image-text alignment, lacking a deep contextual understanding of user interests. To overcome these limitations, we propose an agent-centric personalized clustering framework that leverages multi-modal large language models (MLLMs) as agents to comprehensively traverse a relational graph to search for clusters based on user interests. Due to the advanced reasoning mechanism of MLLMs, the obtained clusters align more closely with user-defined criteria than those obtained from CLIP-based representations. To reduce computational overhead, we shorten the agents' traversal path by constructing a relational graph using user-interest-biased embeddings extracted by MLLMs. A large number of weakly connected edges can be filtered out based on embedding similarity, facilitating an efficient traversal search for agents. Experimental results show that the proposed method achieves NMI scores of 0.9667 and 0.9481 on the Card Order and Card Suits benchmarks, respectively, largely improving the SOTA model by over 140%.
Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
Hoang, Cong Duy Vu, Tangari, Gioacchino, Lanfranchi, Clemence, Guo, Dalu, Cayet, Paul, Siu, Steve, Dharmasiri, Don, Li, Yuan-Fang, Duong, Long, Hilloulin, Damien, Patra, Rhicheek, Hong, Sungpack, Chafi, Hassan
The growing adoption of large language models (LLMs) in business applications has amplified interest in Natural Language to SQL (NL2SQL) solutions, in which there is competing demand for high performance and efficiency. Domain- and customer-specific requirements further complicate the problem. To address this conundrum, we introduce Distill-C, a distilled customization framework tailored for NL2SQL tasks. Distill-C utilizes large teacher LLMs to produce high-quality synthetic data through a robust and scalable pipeline. Finetuning smaller and open-source LLMs on this synthesized data enables them to rival or outperform teacher models an order of magnitude larger. Evaluated on multiple challenging benchmarks, Distill-C achieves an average improvement of 36% in execution accuracy compared to the base models from three distinct LLM families. Additionally, on three internal customer benchmarks, Distill-C demonstrates a 22.6% performance improvement over the base models. Our results demonstrate that Distill-C is an effective, high-performing and generalizable approach for deploying lightweight yet powerful NL2SQL models, delivering exceptional accuracies while maintaining low computational cost.
MiZero: The Shadowy Defender Against Text Style Infringements
Zhang, Ziwei, Wen, Juan, Peng, Wanli, Wu, Zhengxian, Zhou, Yinghan, Xue, Yiming
In-Context Learning (ICL) and efficient fine-tuning methods significantly enhanced the efficiency of applying Large Language Models (LLMs) to downstream tasks. However, they also raise concerns about the imitation and infringement of personal creative data. Current methods for data copyright protection primarily focuses on content security but lacks effectiveness in protecting the copyrights of text styles. In this paper, we introduce a novel implicit zero-watermarking scheme, namely MiZero. This scheme establishes a precise watermark domain to protect the copyrighted style, surpassing traditional watermarking methods that distort the style characteristics. Specifically, we employ LLMs to extract condensed-lists utilizing the designed instance delimitation mechanism. These lists guide MiZero in generating the watermark. Extensive experiments demonstrate that MiZero effectively verifies text style copyright ownership against AI imitation.