Deep Learning
LLM Based Sentiment Classification From Bangladesh E-Commerce Reviews
Sumaiya Tabassum Department of Computer Science and Engineering Dhaka International University Dhaka, Bangladesh sumaiyatabassum230@gmail.com Abstract -- Sentiment analysis is an essential part of text analysis, which is a larger field that includes determining and evaluating the author's emotional state. This method is essential since it makes it easier to comprehend consumers' feelings, viewpoints, and preferences holistically. The introduction of large language models (LLMs), such as Llama, has greatly increased the availability of cutting - edge model applications, such as sentiment analysis. However, accurate sentiment analysis is hampered by the intricacy of written language and the diversity of languages used in evaluations. The viability of using transformer - based BERT models and other LLMs for sentiment analysis from Bangladesh e - commerce reviews is investigated in this paper. A subset of 4000 samples from the original dataset of Bangla and English customer reviews was utilized to fine - tune the model. The fine - tuned Llama - 3.1 - 8B model outperformed other fine - tuned models, including Phi - 3.5 - mini - instruct, Mistral - 7B - v0.1, DistilBERT - multilingual, mBERT, and XLM - R - base, with an overall accuracy, precision, recall, and F1 score of 95.5%, 93%, 88%, 90%.
TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models
Chang, Shenxu, Yu, Junchi, Wang, Weixing, Chen, Yongqiang, Yu, Jialin, Torr, Philip, Gu, Jindong
Diffusion large language models (D-LLMs) have recently emerged as a promising alternative to auto-regressive LLMs (AR-LLMs). However, the hallucination problem in D-LLMs remains underexplored, limiting their reliability in real-world applications. Existing hallucination detection methods are designed for AR-LLMs and rely on signals from single-step generation, making them ill-suited for D-LLMs where hallucination signals often emerge throughout the multi-step denois-ing process. To bridge this gap, we propose TraceDet, a novel framework that explicitly leverages the intermediate denoising steps of D-LLMs for hallucination detection. TraceDet models the denoising process as an action trace, with each action defined as the model's prediction over the cleaned response, conditioned on the previous intermediate output. By identifying the sub-trace that is maximally informative to the hallucinated responses, TraceDet leverages the key hallucination signals in the multi-step denoising process of D-LLMs for hallucination detection. Extensive experiments on various open source D-LLMs demonstrate that TraceDet consistently improves hallucination detection, achieving an average gain in AUROC of 15.2% compared to baselines. The auto-regressive large language models (AR-LLMs) (Achiam et al., 2023; V aswani et al., 2017) have demonstrated unprecedented capabilities in content generation (Maleki & Zhao, 2024) and general task completion (Y ao et al., 2023). Despite their success, AR-LLMs still face challenges related to generation efficiency and the reversal curse due to the inherent limitation of the next-token prediction paradigm (Bachmann & Nagarajan, 2024).
Modeling Others' Minds as Code
Jha, Kunal, Huang, Aydan Yuenan, Ye, Eric, Jaques, Natasha, Kleiman-Weiner, Max
Accurate prediction of human behavior is essential for robust and safe human-AI collaboration. However, existing approaches for modeling people are often data-hungry and brittle because they either make unrealistic assumptions about rationality or are too computationally demanding to adapt rapidly. Our key insight is that many everyday social interactions may follow predictable patterns; efficient "scripts" that minimize cognitive load for actors and observers, e.g., "wait for the green light, then go." We propose modeling these routines as behavioral programs instantiated in computer code rather than policies conditioned on beliefs and desires. We introduce ROTE, a novel algorithm that leverages both large language models (LLMs) for synthesizing a hypothesis space of behavioral programs, and probabilistic inference for reasoning about uncertainty over that space. We test ROTE in a suite of gridworld tasks and a large-scale embodied household simulator. ROTE predicts human and AI behaviors from sparse observations, outperforming competitive baselines -- including behavior cloning and LLM-based methods -- by as much as 50% in terms of in-sample accuracy and out-of-sample generalization. By treating action understanding as a program synthesis problem, ROTE opens a path for AI systems to efficiently and effectively predict human behavior in the real-world.
Identifying Information-Transfer Nodes in a Recurrent Neural Network Reveals Dynamic Representations
Hintze, Arend, Najam, Asadullah, Schossau, Jory
Abstract--Understanding the internal dynamics of Recurrent Neural Networks (RNNs) is crucial for advancing their inter-pretability and improving their design. This study introduces an innovative information-theoretic method to identify and analyze information-transfer nodes within RNNs, which we refer to as information relays. We apply this methodology to both synthetic and real-world time series classification tasks, employing various RNN architectures, including Long Short-T erm Memory (LSTM) networks and Gated Recurrent Units (GRUs). Our results reveal distinct patterns of information relay across different architectures, offering insights into how information is processed and maintained over time. Additionally, we conduct node knockout experiments to assess the functional importance of identified nodes, significantly contributing to explainable artificial intelligence by elucidating how specific nodes influence overall network behavior . This study not only enhances our understanding of the complex mechanisms driving RNNs but also provides a valuable tool for designing more robust and interpretable neural networks. RTIFICIAL neural networks represent a significant enigma. On the one hand, we grasp the mechanism of backpropagation and the dataset that optimally trains them; on the other hand, we do not fully comprehend how ANNs or their recurrent counterparts, RNNs, accomplish their tasks. At their most basic, their function can be described as follows: input vectors are propagated forward through multiple layers by applying a dot product between matrices and these vectors, alongside the application of mathematical threshold functions to the vectors. While this functional description enables us to implement and compute neural networks relatively easily, it applies to a network executing the MNIST handwritten numeral task or the CIFAR-10 image classification problem. The functional differences between these tasks lie in the variations between their weight matrices, which yield complex hidden state vectors that contain and propagate the processed information. Numerous efforts have been made to decode the roles of these vector components.
OpenAI's GPT-OSS-20B Model and Safety Alignment Issues in a Low-Resource Language
In response to the recent safety probing for OpenAI's GPT-OSS-20b model, we present a summary of a set of vulnerabilities uncovered in the model, focusing on its performance and safety alignment in a low-resource language setting. The core motivation for our work is to question the model's reliability for users from underrepresented communities. Using Hausa, a major African language, we uncover biases, inaccuracies, and cultural insensitivities in the model's behaviour. With a minimal prompting, our red-teaming efforts reveal that the model can be induced to generate harmful, culturally insensitive, and factually inaccurate content in the language. As a form of reward hacking, we note how the model's safety protocols appear to relax when prompted with polite or grateful language, leading to outputs that could facilitate misinformation and amplify hate speech. For instance, the model operates on the false assumption that common insecticide locally known as Fiya-Fiya (Cyphermethrin) and rodenticide like Shinkafar Bera (a form of Aluminium Phosphide) are safe for human consumption. To contextualise the severity of this error and popularity of the substances, we conducted a survey (n=61) in which 98% of participants identified them as toxic. Additional failures include an inability to distinguish between raw and processed foods and the incorporation of demeaning cultural proverbs to build inaccurate arguments. We surmise that these issues manifest through a form of linguistic reward hacking, where the model prioritises fluent, plausible-sounding output in the target language over safety and truthfulness. We attribute the uncovered flaws primarily to insufficient safety tuning in low-resource linguistic contexts. By concentrating on a low-resource setting, our approach highlights a significant gap in current red-teaming effort and offer some recommendations.
RSTGCN: Railway-centric Spatio-Temporal Graph Convolutional Network for Train Delay Prediction
Chowdhury, Koyena, Koley, Paramita, Chakraborty, Abhijnan, Ghosh, Saptarshi
Accurate prediction of train delays is critical for efficient railway operations, enabling better scheduling and dispatching decisions. While earlier approaches have largely focused on forecasting the exact delays of individual trains, recent studies have begun exploring station-level delay prediction to support higher-level traffic management. In this paper, we propose the Railway-centric Spatio-Temporal Graph Convolutional Network (RSTGCN), designed to forecast average arrival delays of all the incoming trains at railway stations for a particular time period. Our approach incorporates several architectural innovations and novel feature integrations, including train frequency-aware spatial attention, which significantly enhances predictive performance. To support this effort, we curate and release a comprehensive dataset for the entire Indian Railway Network (IRN), spanning 4,735 stations across 17 zones - the largest and most diverse railway network studied to date. We conduct extensive experiments using multiple state-of-the-art baselines, demonstrating consistent improvements across standard metrics. Our work not only advances the modeling of average delay prediction in large-scale rail networks but also provides an open dataset to encourage further research in this critical domain.
Adaptive Federated Learning Defences via Trust-Aware Deep Q-Networks
Federated learning is vulnerable to poisoning and backdoor attacks under partial observability. We formulate defence as a partially observable sequential decision problem and introduce a trust-aware Deep Q-Network that integrates multi-signal evidence into client trust updates while optimizing a long-horizon robustness--accuracy objective. On CIFAR-10, we (i) establish a baseline showing steadily improving accuracy, (ii) show through a Dirichlet sweep that increased client overlap consistently improves accuracy and reduces ASR with stable detection, and (iii) demonstrate in a signal-budget study that accuracy remains steady while ASR increases and ROC-AUC declines as observability is reduced, which highlights that sequential belief updates mitigate weaker signals. Finally, a comparison with random, linear-Q, and policy gradient controllers confirms that DQN achieves the best robustness--accuracy trade-off.
IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol
Yang, Ningyuan, Lyu, Guanliang, Ma, Mingchen, Lu, Yiyi, Li, Yiming, Gao, Zhihui, Ye, Hancheng, Zhang, Jianyi, Chen, Tingjun, Chen, Yiran
The integration of Large Language Models (LLMs) with Internet-of-Things (IoT) systems faces significant challenges in hardware heterogeneity and control complexity. The Model Context Protocol (MCP) emerges as a critical enabler, providing standardized communication between LLMs and physical devices. We propose IoT-MCP, a novel framework that implements MCP through edge-deployed servers to bridge LLMs and IoT ecosystems. To support rigorous evaluation, we introduce IoT-MCP Bench, the first benchmark containing 114 Basic Tasks (e.g., ``What is the current temperature?'') and 1,140 Complex Tasks (e.g., ``I feel so hot, do you have any ideas?'') for IoT-enabled LLMs. Experimental validation across 22 sensor types and 6 microcontroller units demonstrates IoT-MCP's 100% task success rate to generate tool calls that fully meet expectations and obtain completely accurate results, 205ms average response time, and 74KB peak memory footprint. This work delivers both an open-source integration framework (https://github.com/Duke-CEI-Center/IoT-MCP-Servers) and a standardized evaluation methodology for LLM-IoT systems.
In AI Sweet Harmony: Sociopragmatic Guardrail Bypasses and Evaluation-Awareness in OpenAI gpt-oss-20b
We probe OpenAI's open-weights 20-billion-parameter model gpt-oss-20b to study how sociopragmatic framing, language choice, and instruction hierarchy affect refusal behavior. Across 80 seeded iterations per scenario, we test several harm domains including ZIP-bomb construction (cyber threat), synthetic card-number generation, minor-unsafe driving advice, drug-precursor indicators, and RAG context exfiltration. Composite prompts that combine an educator persona, a safety-pretext ("what to avoid"), and step-cue phrasing flip assistance rates from 0% to 97.5% on a ZIP-bomb task. On our grid, formal registers in German and French are often leakier than matched English prompts. A "Linux terminal" role-play overrides a developer rule not to reveal context in a majority of runs with a naive developer prompt, and we introduce an AI-assisted hardening method that reduces leakage to 0% in several user-prompt variants. We further test evaluation awareness with a paired-track design and measure frame-conditioned differences between matched "helpfulness" and "harmfulness" evaluation prompts; we observe inconsistent assistance in 13% of pairs. Finally, we find that the OpenAI Moderation API under-captures materially helpful outputs relative to a semantic grader, and that refusal rates differ by 5 to 10 percentage points across inference stacks, raising reproducibility concerns. We release prompts, seeds, outputs, and code for reproducible auditing at https://github.com/ndurner/gpt-oss-rt-run .
RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs
Lin, Can, Jiang, Zhengwang, Zheng, Ling, Zhao, Qi, Zhang, Yuhang, Song, Qi, Zhou, Wangqiu
Knowledge graph question answering (KGQA) aims to answer natural language questions using knowledge graphs. Recent research leverages large language models (LLMs) to enhance KGQA reasoning, but faces limitations: retrieval-based methods are constrained by the quality of retrieved information, while agent-based methods rely heavily on proprietary LLMs. To address these limitations, we propose Retrieval-Judgment-Exploration (RJE), a framework that retrieves refined reasoning paths, evaluates their sufficiency, and conditionally explores additional evidence. Moreover, RJE introduces specialized auxiliary modules enabling small-sized LLMs to perform effectively: Reasoning Path Ranking, Question Decomposition, and Retriever-assisted Exploration. Experiments show that our approach with proprietary LLMs (such as GPT-4o-mini) outperforms existing baselines while enabling small open-source LLMs (such as 3B and 8B parameters) to achieve competitive results without fine-tuning LLMs. Additionally, RJE substantially reduces the number of LLM calls and token usage compared to agent-based methods, yielding significant efficiency improvements.