Goto

Collaborating Authors

 Large Language Model


Annotate Rhetorical Relations with INCEpTION: A Comparison with Automatic Approaches

arXiv.org Artificial Intelligence

Automatically identifying rhetorical relations in discourse units is a challenging task in natural language processing (NLP) because it should be able to logically and semantically connect the discourse units. Although large language models (LLMs) shows po tential for application in many domains, including text classification tasks, their effectiveness in predicting rhetorical relations remains open for research. One of the major challenges in this domain is the lack of annotated data sets capturing differen t rhetorical relations, which would then make model training more difficult. In this research, we manually created the da-tasets from various cricket reports and then annotated the reports as discourse units. We used the INCEpTION annotation tools for annotation and then structured the dataset for the machine - learning model.


Mechanistic Interpretability of Socio-Political Frames in Language Models

arXiv.org Artificial Intelligence

This paper explores the ability of large language models to generate and recognize deep cognitive frames, particularly in socio-political contexts. We demonstrate that LLMs are highly fluent in generating texts that evoke specific frames and can recognize these frames in zero-shot settings. Inspired by mechanistic interpretability research, we investigate the location of the `strict father' and `nurturing parent' frames within the model's hidden representation, identifying singular dimensions that correlate strongly with their presence. Our findings contribute to understanding how LLMs capture and express meaningful human concepts.


Investigating LLM Variability in Personalized Conversational Information Retrieval

arXiv.org Artificial Intelligence

Personalized Conversational Information Retrieval (CIR) has seen rapid progress in recent years, driven by the development of Large Language Models (LLMs). Personalized CIR aims to enhance document retrieval by leveraging user-specific information, such as preferences, knowledge, or constraints, to tailor responses to individual needs. A key resource for this task is the TREC iKAT 2023 dataset, designed to evaluate personalization in CIR pipelines. Building on this resource, Mo et al. explored several strategies for incorporating Personal Textual Knowledge Bases (PTKB) into LLM-based query reformulation. Their findings suggested that personalization from PTKBs could be detrimental and that human annotations were often noisy. However, these conclusions were based on single-run experiments using the GPT-3.5 Turbo model, raising concerns about output variability and repeatability. In this reproducibility study, we rigorously reproduce and extend their work, focusing on LLM output variability and model generalization. We apply the original methods to the new TREC iKAT 2024 dataset and evaluate a diverse range of models, including Llama (1B-70B), Qwen-7B, GPT-4o-mini. Our results show that human-selected PTKBs consistently enhance retrieval performance, while LLM-based selection methods do not reliably outperform manual choices. We further compare variance across datasets and observe higher variability on iKAT than on CAsT, highlighting the challenges of evaluating personalized CIR. Notably, recall-oriented metrics exhibit lower variance than precision-oriented ones, a critical insight for first-stage retrievers. Finally, we underscore the need for multi-run evaluations and variance reporting when assessing LLM-based CIR systems. By broadening evaluation across models, datasets, and metrics, our study contributes to more robust and generalizable practices for personalized CIR.


Lightweight and Data-Efficient MultivariateTime Series Forecasting using Residual-Stacked Gaussian (RS-GLinear) Architecture

arXiv.org Artificial Intelligence

-- Following the success of Transformer architectures and their self - attention mechanism in language modelling -- particularly due to their ability to capture long - range dependencies -- many researchers have explored how these architectures can be adopted for time - series forecasting. Varia nts of Transformer - based models have been proposed to handle both short - and long - term sequence modeling, aiming to predict future time - dependent values from historical observations using varying input window sizes. However, despite the popularity of lever a ging Transformer architecture to extract temporal relationships from set of continu ou s datapoints, their performance in time - series forecasting has shown mixed results. Several researchers, including Zeng et al. (2022) and Rizvi et al. (2025), have challenged the reliability of emerging Transformer - based solutions for long - term forecasting tasks. In this research, our first objective is to evaluate the G aussian - based Linear (GLinear) architecture proposed by Ri z vi et al. (2025) and to develop an enhanced ve rsion of it -- referred to in this study as Residual Stacked GLinear (RS - GLinear) model. The second objective is to assess the broader applicability of the RS - GLinear model by extending its use to additional domain -- financial time series and epidemiological data -- which were not explored in the baseline model proposed by Rizvi et al. (2025). Most time - series implementations (Transformer - based and Linear models) we came across commonly adopt baseline codebases provided by the Hugging Face repository, including our baseline GLinear model used in this study. Therefore, the RS - GLinear model developed in this study is an extended version of the codebase introduced in the research by Rizvi et al. (2025) . Keywords -- Multivariate Time Series Forecasting, Transformer - based models, Weather, Influenza - like Illness, Deep Learning, Transformer - based architecture, Residual - Stacked GLinear, Neural - Network. Time series forecasting has been an important research area in many domains such as finance/economics, retail, healthcare, cloud infrastructure, met eo rology, and traffic management (Toner e t al. 2024). Since the introduction of T ransformer Model (Vaswani et al. 2017), there has been large amount of research focusing on time - series forecasting using Large Language Models (LLM) to leverage LLM's sequential dependencies in text generation (Tan et al. 2024).


Rezwan: Leveraging Large Language Models for Comprehensive Hadith Text Processing: A 1.2M Corpus Development

arXiv.org Artificial Intelligence

This paper presents the development of Rezwan, a large-scale AI-assisted Hadith corpus comprising over 1.2M narrations, extracted and structured through a fully automated pipeline. Building on digital repositories such as Maktabat Ahl al-Bayt, the pipeline employs Large Language Models (LLMs) for segmentation, chain--text separation, validation, and multi-layer enrichment. Each narration is enhanced with machine translation into twelve languages, intelligent diacritization, abstractive summarization, thematic tagging, and cross-text semantic analysis. This multi-step process transforms raw text into a richly annotated research-ready infrastructure for digital humanities and Islamic studies. A rigorous evaluation was conducted on 1,213 randomly sampled narrations, assessed by six domain experts. Results show near-human accuracy in structured tasks such as chain--text separation (9.33/10) and summarization (9.33/10), while highlighting ongoing challenges in diacritization and semantic similarity detection. Comparative analysis against the manually curated Noor Corpus demonstrates the superiority of Najm in both scale and quality, with a mean overall score of 8.46/10 versus 3.66/10. Furthermore, cost analysis confirms the economic feasibility of the AI approach: tasks requiring over 229,000 hours of expert labor were completed within months at a fraction of the cost. The work introduces a new paradigm in religious text processing by showing how AI can augment human expertise, enabling large-scale, multilingual, and semantically enriched access to Islamic heritage.


OptAgent: Optimizing Query Rewriting for E-commerce via Multi-Agent Simulation

arXiv.org Artificial Intelligence

Deploying capable and user-aligned LLM-based systems necessitates reliable evaluation. While LLMs excel in verifiable tasks like coding and mathematics, where gold-standard solutions are available, adoption remains challenging for subjective tasks that lack a single correct answer. E-commerce Query Rewriting (QR) is one such problem where determining whether a rewritten query properly captures the user intent is extremely difficult to figure out algorithmically. In this work, we introduce OptAgent, a novel framework that combines multi-agent simulations with genetic algorithms to verify and optimize queries for QR. Instead of relying on a static reward model or a single LLM judge, our approach uses multiple LLM-based agents, each acting as a simulated shopping customer, as a dynamic reward signal. The average of these agent-derived scores serves as an effective fitness function for an evolutionary algorithm that iteratively refines the user's initial query. We evaluate OptAgent on a dataset of 1000 real-world e-commerce queries in five different categories, and we observe an average improvement of 21.98% over the original user query and 3.36% over a Best-of-N LLM rewriting baseline.


Prompt Balance Matters: Understanding How Imbalanced Few-Shot Learning Affects Multilingual Sense Disambiguation in LLMs

arXiv.org Artificial Intelligence

Recent advances in Large Language Models (LLMs) have significantly reshaped the landscape of Natural Language Processing (NLP). Among the various prompting techniques, few-shot prompting has gained considerable attention for its practicality and effectiveness. This study investigates how few-shot prompting strategies impact the Word Sense Disambiguation (WSD) task, particularly focusing on the biases introduced by imbalanced sample distributions. We use the GLOSSGPT prompting method, an advanced approach for English WSD, to test its effectiveness across five languages: English, German, Spanish, French, and Italian. Our results show that imbalanced few-shot examples can cause incorrect sense predictions in multilingual languages, but this issue does not appear in English. To assess model behavior, we evaluate both the GPT-4o and LLaMA-3.1-70B models and the results highlight the sensitivity of multilingual WSD to sample distribution in few-shot settings, emphasizing the need for balanced and representative prompting strategies.


You Have Been LaTeXpOsEd: A Systematic Analysis of Information Leakage in Preprint Archives Using Large Language Models

arXiv.org Artificial Intelligence

The widespread use of preprint repositories such as arXiv has accelerated the communication of scientific results but also introduced overlooked security risks. Beyond PDFs, these platforms provide unrestricted access to original source materials, including LaTeX sources, auxiliary code, figures, and embedded comments. In the absence of sanitization, submissions may disclose sensitive information that adversaries can harvest using open-source intelligence. In this work, we present the first large-scale security audit of preprint archives, analyzing more than 1.2 TB of source data from 100,000 arXiv submissions. We introduce LaTeXpOsEd, a four-stage framework that integrates pattern matching, logical filtering, traditional harvesting techniques, and large language models (LLMs) to uncover hidden disclosures within non-referenced files and LaTeX comments. To evaluate LLMs' secret-detection capabilities, we introduce LLMSec-DB, a benchmark on which we tested 25 state-of-the-art models. Our analysis uncovered thousands of PII leaks, GPS-tagged EXIF files, publicly available Google Drive and Dropbox folders, editable private SharePoint links, exposed GitHub and Google credentials, and cloud API keys. We also uncovered confidential author communications, internal disagreements, and conference submission credentials, exposing information that poses serious reputational risks to both researchers and institutions. We urge the research community and repository operators to take immediate action to close these hidden security gaps. To support open science, we release all scripts and methods from this study but withhold sensitive findings that could be misused, in line with ethical principles. The source code and related material are available at the project website https://github.com/LaTeXpOsEd


EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models

arXiv.org Artificial Intelligence

CUDA kernel optimization has become a critical bottleneck for AI performance, as deep learning training and inference efficiency directly depends on highly optimized GPU kernels. Despite the promise of Large Language Models (LLMs) for automating kernel optimization, this field suffers from a fragmented ecosystem of isolated and incomparable approaches with unclear problem formulations. Furthermore, general-purpose LLM code evolution methods cannot meet strict correctness requirements of CUDA kernel optimization. We address these fundamental challenges by first formalizing CUDA kernel optimization as a code optimization task with a clear objective, constraints, and evaluation metrics. We then establish the first systematic LLM-based code evolution framework, EvoEngineer, that provides guidance for designing and adapting optimization strategies to achieve a balance between performance and correctness. Finally, we implement a kernel optimization system based on this framework and conduct extensive experiments on 91 real-world CUDA kernels. Our results demonstrate that EvoEngineer achieves a principled balance between performance and correctness, with the highest averaged median speedup of 2.72 over baseline CUDA kernels and a code validity rate of 69.8%, outperforming existing methods on both dimensions. Our method achieves a maximum speedup of 36.75 among all operations over PyTorch kernels and delivers the highest speedup on 28 (56.0%) of 50 operations that achieve over 2 acceleration. CUDA kernel performance has become the critical bottleneck constraining the efficiency of AI training and inference. As foundation models continue scaling to unprecedented sizes (Guo et al., 2025; Jaech et al., 2024), computational demands necessitate maximum GPU utilization efficiency, where even marginal improvements in kernel performance can yield substantial reductions in computational costs. However, manual kernel optimization requires deep expertise across GPU architectures, memory hierarchies, parallelization patterns, and hardware-specific features (Navarro et al., 2020; Hennessy & Patterson, 2011), constituting a major obstacle to scaling AI systems. The kernel code optimization landscape presents extreme complexity, involving intricate tradeoffs between memory coalescing, thread divergence, occupancy optimization, and register usage (Ujald on, 2016; Huang et al., 2021; Zhao et al., 2022).


Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models

arXiv.org Artificial Intelligence

Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M. We address this gap by creating person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions. These annotations are produced through validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers. Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content. We also show that 60-70% of gender bias in CLIP and Stable Diffusion can be linearly explained by direct co-occurrences in the data. Our resources establish the first large-scale empirical link between dataset composition and downstream model bias.