Deep Learning
From Silent Signals to Natural Language: A Dual-Stage Transformer-LLM Approach
Instead of relying on acoustic signals, SSIs exploit non-vocal modalities that capture articulatory and physiological activity underlying speech production. A variety of input sources have been investigated, including surface electromyography (EMG), ultrasound tongue imaging (UTI), electromagnetic articulography (EMA), and visual cues from lip movements. These modalities offer complementary perspectives on articulatory dynamics, positioning SSIs as a promising solution for communication in scenarios where acoustic speech is impaired. Recent advances in SSI have introduced convolutionaltransducer models combined with transformer encoders and auxiliary phoneme prediction losses, achieving notable improvements in intelligibility. The input signals typically undergo feature extraction to capture articulatory or physiological attributes that preserve discriminative information necessary for speech generation.
Greener Deep Reinforcement Learning: Analysis of Energy and Carbon Efficiency Across Atari Benchmarks
Gardner, Jason, Dutta, Ayan, Roy, Swapnoneel, Kreidl, O. Patrick, Boloni, Ladislau
The growing computational demands of deep reinforcement learning (DRL) have raised concerns about the environmental and economic costs of training large-scale models. While algorithmic efficiency in terms of learning performance has been extensively studied, the energy requirements, greenhouse gas emissions, and monetary costs of DRL algorithms remain largely unexplored. In this work, we present a systematic benchmarking study of the energy consumption of seven state-of-the-art DRL algorithms, namely DQN, TRPO, A2C, ARS, PPO, RecurrentPPO, and QR-DQN, implemented using Stable Baselines. Each algorithm was trained for one million steps each on ten Atari 2600 games, and power consumption was measured in real-time to estimate total energy usage, CO2-Equivalent emissions, and electricity cost based on the U.S. national average electricity price. Our results reveal substantial variation in energy efficiency and training cost across algorithms, with some achieving comparable performance while consuming up to 24% less energy (ARS vs. DQN), emitting nearly 68% less CO2, and incurring almost 68% lower monetary cost (QR-DQN vs. RecurrentPPO) than less efficient counterparts. We further analyze the trade-offs between learning performance, training time, energy use, and financial cost, highlighting cases where algorithmic choices can mitigate environmental and economic impact without sacrificing learning performance. This study provides actionable insights for developing energy-aware and cost-efficient DRL practices and establishes a foundation for incorporating sustainability considerations into future algorithmic design and evaluation.
On Evaluating the Poisoning Robustness of Federated Learning under Local Differential Privacy
Wang, Zijian, Tong, Wei, Han, Tingxuan, Chen, Haoyu, Zhang, Tianling, Mao, Yunlong, Zhong, Sheng
Abstract--Federated learning (FL) combined with local differential privacy (LDP) enables privacy-preserving model training across decentralized data sources. However, the decentralized data-management paradigm leaves LDPFL vulnerable to participants with malicious intent. The robustness of LDPFL protocols, particularly against model poisoning attacks (MPA), where adversaries inject malicious updates to disrupt global model convergence, remains insufficiently studied. In this paper, we propose a novel and extensible model poisoning attack framework tailored for LDPFL settings. Our approach is driven by the objective of maximizing the global training loss while adhering to local privacy constraints. T o counter robust aggregation mechanisms such as Multi-Krum and trimmed mean, we develop adaptive attacks that embed carefully crafted constraints into a reverse training process, enabling evasion of these defenses. We evaluate our framework across three representative LDPFL protocols, three benchmark datasets, and two types of deep neural networks. Additionally, we investigate the influence of data heterogeneity and privacy budgets on attack effectiveness. Experimental results demonstrate that our adaptive attacks can significantly degrade the performance of the global model, revealing critical vulnerabilities and highlighting the need for more robust LDPFL defense strategies against MPA. Index T erms--federated learning, locally differential privacy, poisoning robustness I. INTRODUCTION Federated Learning (FL) [1]-[8] has become a powerful paradigm enabling collaborative model training across distributed clients without direct access to clients' local datasets. It has been adopted in various applications, from improving input prediction in personalized keyboards [1] and advancing disease prediction [2], to predicting spatio-temporal traffic series [3] and enhancing trajectory similarity calculation [4].
COGITAO: A Visual Reasoning Framework To Study Compositionality & Generalization
Taoudi-Benchekroun, Yassine, Troyan, Klim, Sager, Pascal, Gerber, Stefan, Tuggener, Lukas, Grewe, Benjamin
The ability to compose learned concepts and apply them in novel settings is key to human intelligence, but remains a persistent limitation in state-of-the-art machine learning models. To address this issue, we introduce COGITAO, a modular and extensible data generation framework and benchmark designed to systematically study compositionality and generalization in visual domains. Drawing inspiration from ARC-AGI's problem-setting, COGITAO constructs rule-based tasks which apply a set of transformations to objects in grid-like environments. It supports composition, at adjustable depth, over a set of 28 interoperable transformations, along with extensive control over grid parametrization and object properties. This flexibility enables the creation of millions of unique task rules -- surpassing concurrent datasets by several orders of magnitude -- across a wide range of difficulties, while allowing virtually unlimited sample generation per rule. We provide baseline experiments using state-of-the-art vision models, highlighting their consistent failures to generalize to novel combinations of familiar elements, despite strong in-domain performance. COGITAO is fully open-sourced, including all code and datasets, to support continued research in this field.
Uncertain but Useful: Leveraging CNN Variability into Data Augmentation
Gonzalez-Pepe, Inรฉs, Sivakolunthu, Vinuyan, Chatelain, Yohan, Glatard, Tristan
Deep learning (DL) is rapidly advancing neuroimaging by achieving state-of-the-art performance with reduced computation times. Yet the numerical stability of DL models -- particularly during training -- remains underexplored. While inference with DL is relatively stable, training introduces additional variability primarily through iterative stochastic optimization. We investigate this training-time variability using FastSurfer, a CNN-based whole-brain segmentation pipeline. Controlled perturbations are introduced via floating point perturbations and random seeds. We find that: (i) FastSurfer exhibits higher variability compared to that of a traditional neuroimaging pipeline, suggesting that DL inherits and is particularly susceptible to sources of instability present in its predecessors; (ii) ensembles generated with perturbations achieve performance similar to an unperturbed baseline; and (iii) variability effectively produces ensembles of numerical model families that can be repurposed for downstream applications. As a proof of concept, we demonstrate that numerical ensembles can be used as a data augmentation strategy for brain age regression. These findings position training-time variability not only as a reproducibility concern but also as a resource that can be harnessed to improve robustness and enable new applications in neuroimaging.
Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
Waheed, Abdul, Mitra, Chancharik, Wang, Laurie Z., Ramanan, Deva, Raj, Bhiksha
Chain-of-thought reasoning, while powerful, can produce unnecessarily verbose output for simpler problems. We present a framework for difficulty-aware reasoning that teaches models to dynamically adjust reasoning depth based on problem complexity. Remarkably, we show that models can be endowed with such dynamic inference pathways without any architectural modifications; we simply post-train on data that is carefully curated to include chain-of-thought traces that are proportional in length to problem difficulty. Our analysis reveals that post-training via supervised fine-tuning (SFT) primarily captures patterns like reasoning length and format, while direct preference optimization (DPO) preserves reasoning accuracy, with their combination reducing length and maintaining or improving performance. Both quantitative metrics and qualitative assessments confirm that models can learn to "think proportionally", reasoning minimally on simple problems while maintaining depth for complex ones.
An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
Zhang, Jiaojiao, Xu, Yuqi, Yuan, Kun
This work addresses the key challenges of applying federated learning to large-scale deep neural networks, particularly the issue of client drift due to data heterogeneity across clients and the high costs of communication, computation, and memory. We propose FedSub, an efficient subspace algorithm for federated learning on heterogeneous data. Specifically, FedSub utilizes subspace projection to guarantee local updates of each client within low-dimensional subspaces, thereby reducing communication, computation, and memory costs. Additionally, it incorporates low-dimensional dual variables to mitigate client drift. We provide convergence analysis that reveals the impact of key factors such as step size and subspace projection matrices on convergence. Experimental results demonstrate its efficiency.
Symbolic Graphics Programming with Large Language Models
Chen, Yamei, Zhang, Haoquan, Huang, Yangyi, Qiu, Zeju, Zhang, Kaipeng, Wen, Yandong, Liu, Weiyang
Large language models (LLMs) excel at program synthesis, yet their ability to produce symbolic graphics programs (SGPs) that render into precise visual content remains underexplored. We study symbolic graphics programming, where the goal is to generate an SGP from a natural-language description. This task also serves as a lens into how LLMs understand the visual world by prompting them to generate images rendered from SGPs. Among various SGPs, our paper sticks to scalable vector graphics (SVGs). We begin by examining the extent to which LLMs can generate SGPs. To this end, we introduce SGP-GenBench, a comprehensive benchmark covering object fidelity, scene fidelity, and compositionality (attribute binding, spatial relations, numeracy). On SGP-GenBench, we discover that frontier proprietary models substantially outperform open-source models, and performance correlates well with general coding capabilities. Motivated by this gap, we aim to improve LLMs' ability to generate SGPs. We propose a reinforcement learning (RL) with verifiable rewards approach, where a format-validity gate ensures renderable SVG, and a cross-modal reward aligns text and the rendered image via strong vision encoders (e.g., SigLIP for text-image and DINO for image-image). Applied to Qwen-2.5-7B, our method substantially improves SVG generation quality and semantics, achieving performance on par with frontier systems. We further analyze training dynamics, showing that RL induces (i) finer decomposition of objects into controllable primitives and (ii) contextual details that improve scene coherence. Our results demonstrate that symbolic graphics programming offers a precise and interpretable lens on cross-modal grounding.
Triadic Fusion of Cognitive, Functional, and Causal Dimensions for Explainable LLMs: The TAXAL Framework
Herrera-Poyatos, David, Pelรกez-Gonzรกlez, Carlos, Zuheros, Cristina, Tejedor, Virilo, Montes, Rosana, Herrera, Francisco
Large Language Models (LLMs) such as GPT -5, GEMINI, Claude, and LLaMA have become foundational tools in artificial intelligence (AI), achieving state-of-the-art performance in summarization, translation, reasoning, and dialogue. However, since LLMs are increasingly integrated in high-risk decision making in domains such as healthcare, law, and education, their lack of transparency raises urgent concerns for safety, accountability, and public trust [12]. The scale and complexity of these models, covering billions of parameters trained in opaque corpora, make their internal reasoning fundamentally inscrutable. This opacity creates barriers to responsible adoption, as users often lack meaningful ways to understand or challenge outputs. Without stakeholder-sensitive explanations, systems risk overtrust, misinterpretation, or outright rejection [11]. Explainable AI (XAI) for LLMs has therefore evolved beyond technical introspection [6]. The goal is not only to expose internal mechanisms but also to support human interaction, trust calibration, and decision assurance. As model behavior becomes more emergent and unpredictable [10], explanation systems must serve cognitive, functional, and ethical purposes simultaneously [7].
Enhancing 3D Point Cloud Classification with ModelNet-R and Point-SkipNet
Saeid, Mohammad, Salarpour, Amir, MohajerAnsari, Pedram
The classification of 3D point clouds is crucial for applications such as autonomous driving, robotics, and augmented reality. However, the commonly used ModelNet40 dataset suffers from limitations such as inconsistent labeling, 2D data, size mismatches, and inadequate class differentiation, which hinder model performance. This paper introduces ModelNet-R, a meticulously refined version of ModelNet40 designed to address these issues and serve as a more reliable benchmark. Additionally, this paper proposes Point-SkipNet, a lightweight graph-based neural network that leverages efficient sampling, neighborhood grouping, and skip connections to achieve high classification accuracy with reduced computational overhead. Extensive experiments demonstrate that models trained in ModelNet-R exhibit significant performance improvements. Notably, Point-SkipNet achieves state-of-the-art accuracy on ModelNet-R with a substantially lower parameter count compared to contemporary models. This research highlights the crucial role of dataset quality in optimizing model efficiency for 3D point cloud classification. For more details, see the code at: https://github.com/m-saeid/ModeNetR_PointSkipNet.