Deep Learning
TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla
Raihan, Nishat, Anastasopoulos, Antonios, Zampieri, Marcos
Despite being the 5th most spoken language, Bangla remains underrepresented in Large Language Models (LLMs), particularly for code generation. This primarily stems from the scarcity of high-quality data to pre-train and/or finetune such models. Hence, we introduce the first dedicated family of Code LLMs for Bangla (1B & 9B). We offer three major contributions: (1) a comprehensive Bangla code instruction datasets for programming domain adaptation; (2) MBPP-Bangla, an evaluation benchmark for Bangla code generation; and (3) the TigerCoder-family of Code LLMs, achieving significant ~11-18% performance gains at Pass@1 over existing multilingual and general-purpose Bangla LLMs. Our findings show that curated, high-quality datasets can overcome limitations of smaller models for low-resource languages. We open-source all resources to advance further Bangla LLM research.
Towards Confidential and Efficient LLM Inference with Dual Privacy Protection
Yu, Honglan, Wang, Yibin, Dai, Feifei, Liu, Dong, Fan, Haihui, Gu, Xiaoyan
CPU-based trusted execution environments (TEEs) and differential privacy (DP) have gained wide applications for private inference. Due to high inference latency in TEEs, researchers use partition-based approaches that offload linear model components to GPUs. However, dense nonlinear layers of large language models (LLMs) result in significant communication overhead between TEEs and GPUs. DP-based approaches apply random noise to protect data privacy, but this compromises LLM performance and semantic understanding. To overcome the above drawbacks, this paper proposes CMIF, a Confidential and efficient Model Inference Framework. CMIF confidentially deploys the embedding layer in the client-side TEE and subsequent layers on GPU servers. Meanwhile, it optimizes the Report-Noisy-Max mechanism to protect sensitive inputs with a slight decrease in model performance. Extensive experiments on Llama-series models demonstrate that CMIF reduces additional inference overhead in TEEs while preserving user data privacy.
MR-UIE: Multi-Perspective Reasoning with Reinforcement Learning for Universal Information Extraction
Li, Zhongqiu, Wang, Shiquan, Fang, Ruiyu, Bao, Mengjiao, Wu, Zhenhe, Song, Shuangyong, Li, Yongxiang, He, Zhongjiang
Information extraction (IE) is a fundamental task in natural language processing (NLP), which encompasses a wide range of subtasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Event Extraction (EE) [1-4]. Traditionally, these tasks have been addressed by specialized models trained in task-specific datasets. However, the fragmentation of tasks and schemas has hindered the development of generalizable and scalable IE tasks. To address this limitation, recent research has focused on universal information extraction (UIE), which aims to model all IE tasks within a universal framework. A seminal work in this direction is proposed by Lu et al., which introduced a structured generation paradigm that encodes diverse IE tasks into a common semantic representation[5]. Building on this, InstructUIE[6] extended the idea by incorporating multi-task instruction tuning, enabling models to generalize across tasks via natural language instructions. With the emergence of powerful LLMs[7-11], significant advancements have been made across long-standing NLP tasks such as text classification[12-16], intent recognition[17, 18], entity linking[19-22], and beyond. Inspired by their robust performance and adaptability, researchers have explored their potential for information extraction through prompting and in-context learning learning[23, 24]. For example, CodeIE demonstrated that code generation models can serve as strong few-shot IE extractors by using structured code-like commands[25].
MoWE : A Mixture of Weather Experts
Chakraborty, Dibyajyoti, Maulik, Romit, Harrington, Peter, Foster, Dallas, Nabian, Mohammad Amin, Choudhry, Sanjay
Data-driven weather models have recently achieved state-of-the-art performance, yet progress has plateaued in recent years. This paper introduces a Mixture of Experts (MoWE) approach as a novel paradigm to overcome these limitations, not by creating a new forecaster, but by optimally combining the outputs of existing models. The MoWE model is trained with significantly lower computational resources than the individual experts. Our model employs a Vision Transformer-based gating network that dynamically learns to weight the contributions of multiple "expert" models at each grid point, conditioned on forecast lead time. This approach creates a synthesized deterministic forecast that is more accurate than any individual component in terms of Root Mean Squared Error (RMSE). Our results demonstrate the effectiveness of this method, achieving up to a 10% lower RMSE than the best-performing AI weather model on a 2-day forecast horizon, significantly outperforming individual experts as well as a simple average across experts. This work presents a computationally efficient and scalable strategy to push the state of the art in data-driven weather prediction by making the most out of leading high-quality forecast models.
Generative quantum advantage for classical and quantum problems
Huang, Hsin-Yuan, Broughton, Michael, Eassa, Norhan, Neven, Hartmut, Babbush, Ryan, McClean, Jarrod R.
Recent breakthroughs in generative machine learning, powered by massive computational resources, have demonstrated unprecedented human-like capabilities. While beyond-classical quantum experiments can generate samples from classically intractable distributions, their complexity has thwarted all efforts toward efficient learning. This challenge has hindered demonstrations of generative quantum advantage: the ability of quantum computers to learn and generate desired outputs substantially better than classical computers. We resolve this challenge by introducing families of generative quantum models that are hard to simulate classically, are efficiently trainable, exhibit no barren plateaus or proliferating local minima, and can learn to generate distributions beyond the reach of classical computers. Using a $68$-qubit superconducting quantum processor, we demonstrate these capabilities in two scenarios: learning classically intractable probability distributions and learning quantum circuits for accelerated physical simulation. Our results establish that both learning and sampling can be performed efficiently in the beyond-classical regime, opening new possibilities for quantum-enhanced generative models with provable advantage.
Fast attention mechanisms: a tale of parallelism
Liu, Jingwen, Yu, Hantao, Sanford, Clayton, Andoni, Alexandr, Hsu, Daniel
Transformers have the representational capacity to simulate Massively Parallel Computation (MPC) algorithms, but they suffer from quadratic time complexity, which severely limits their scalability. We introduce an efficient attention mechanism called Approximate Nearest Neighbor Attention (ANNA) with sub-quadratic time complexity. We prove that ANNA-transformers (1) retain the expressive power previously established for standard attention in terms of matching the capabilities of MPC algorithms, and (2) can solve key reasoning tasks such as Match2 and $k$-hop with near-optimal depth. Using the MPC framework, we further prove that constant-depth ANNA-transformers can simulate constant-depth low-rank transformers, thereby providing a unified way to reason about a broad class of efficient attention approximations.
Value bounds and Convergence Analysis for Averages of LRP attributions
Binder, Alexander, Takmil-Homayouni, Nastaran, Dogan, Urun
We analyze numerical properties of Layer-wise relevance propagation (LRP)-type attribution methods by representing them as a product of modified gradient matrices. This representation creates an analogy to matrix multiplications of Jacobi-matrices which arise from the chain rule of differentiation. In order to shed light on the distribution of attribution values, we derive upper bounds for singular values. Furthermore we derive component-wise bounds for attribution map values. As a main result, we apply these component-wise bounds to obtain multiplicative constants. These constants govern the convergence of empirical means of attributions to expectations of attribution maps. This finding has important implications for scenarios where multiple non-geometric data augmentations are applied to individual test samples, as well as for Smoothgrad-type attribution methods. In particular, our analysis reveals that the constants for LRP-beta remain independent of weight norms, a significant distinction from both gradient-based methods and LRP-epsilon.
FoundationalECGNet: A Lightweight Foundational Model for ECG-based Multitask Cardiac Analysis
Sk., Md. Sajeebul Islam, Jobayer, Md, Shawon, Md Mehedi Hasan, Alam, Md. Golam Raibul
-- Cardiovascular diseases (CVDs) remain a leading cause of mortality worldwide, underscoring the importance of accurate and scalable diagnostic systems. Electrocardiogram (ECG) analysis is central to detecting cardiac abnormalities, yet challenges such as noise, class imbalance, and dataset heterogeneity limit current methods. To address these issues, we propose Foun-dationalECGNet, a foundational framework for automated ECG classification. The model integrates a dual-stage de-noising by Morlet and Daubechies wavelets transformation, Convolutional Block Attention Module (CBAM), Graph Attention Networks (GAT), and Time Series Transformers (TST) to jointly capture spatial and temporal dependencies in multi-channel ECG signals. FoundationalECGNet first distinguishes between Normal and Abnormal ECG signals, and then classifies the Abnormal signals into one of five cardiac conditions: Arrhythmias, Conduction Disorders, Myocardial Infarction, QT Abnormalities, or Hypertrophy. Across multiple datasets, the model achieves a 99% F1-score for Normal vs. Abnormal classification and shows state-of-the-art performance in multi-class disease detection, including a 99% F1-score for Conduction Disorders and Hypertrophy, as well as a 98.9% F1-score for Arrhythmias. Additionally, the model provides risk level estimations to facilitate clinical decision-making. In conclusion, FoundationalECGNet represents a scalable, interpretable, and generalizable solution for automated ECG analysis, with the potential to improve diagnostic precision and patient outcomes in healthcare settings. ARDIOV ASCULAR diseases (CVDs) are the leading cause of death worldwide, leading to approximately 17.9 million deaths each year [45].
BRoverbs -- Measuring how much LLMs understand Portuguese proverbs
Almeida, Thales Sales, Bonás, Giovana Kerche, Santos, João Guilherme Alves
Large Language Models (LLMs) exhibit significant performance variations depending on the linguistic and cultural context in which they are applied. This disparity signals the necessity of mature evaluation frameworks that can assess their capabilities in specific regional settings. In the case of Portuguese, existing evaluations remain limited, often relying on translated datasets that may not fully capture linguistic nuances or cultural references. Meanwhile, native Portuguese-language datasets predominantly focus on structured national exams or sentiment analysis of social media interactions, leaving gaps in evaluating broader linguistic understanding. To address this limitation, we introduce BRoverbs, a dataset specifically designed to assess LLM performance through Brazilian proverbs. Proverbs serve as a rich linguistic resource, encapsulating cultural wisdom, figurative expressions, and complex syntactic structures that challenge the model comprehension of regional expressions. BRoverbs aims to provide a new evaluation tool for Portuguese-language LLMs, contributing to advancing regionally informed benchmarking. The benchmark is available at https://huggingface.co/datasets/Tropic-AI/BRoverbs.
Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities
Haupt, Jarvis, Lu, Qin, Shen, Yanning, Chen, Jia, Dong, Yue, McCreary, Dan, Akçakaya, Mehmet, Giannakis, Georgios B.
Powerful artificial intelligence (AI) tools that have emerged in recent years -- including large language models, automated coding assistants, and advanced image and speech generation technologies -- are the result of monumental human achievements. These breakthroughs reflect mastery across multiple technical disciplines and the resolution of significant technological challenges. However, some of the most profound challenges may still lie ahead. These challenges are not purely technical but pertain to the fair and responsible use of AI in ways that genuinely improve the global human condition. This article explores one promising application aligned with that vision: the use of AI tools to facilitate and enhance education, with a specific focus on signal processing (SP). It presents two interrelated perspectives: identifying and addressing technical limitations, and applying AI tools in practice to improve educational experiences. Primers are provided on several core technical issues that arise when using AI in educational settings, including how to ensure fairness and inclusivity, handle hallucinated outputs, and achieve efficient use of resources. These and other considerations -- such as transparency, explainability, and trustworthiness -- are illustrated through the development of an immersive, structured, and reliable "smart textbook." The article serves as a resource for researchers and educators seeking to advance AI's role in engineering education.