Deep Learning
Revealing Neurocognitive and Behavioral Patterns by Unsupervised Manifold Learning from Dynamic Brain Data
Zhou, Zixia, Liu, Junyan, Wu, Wei Emma, Fang, Ruogu, Liu, Sheng, Wei, Qingyue, Yan, Rui, Guo, Yi, Tao, Qian, Wang, Yuanyuan, Islam, Md Tauhidul, Xing, Lei
Dynamic brain data, teeming with biological and functional insights, are becoming increasingly accessible through advanced measurements, providing a gateway to understanding the inner workings of the brain in living subjects. However, the vast size and intricate complexity of the data also pose a daunting challenge in reliably extracting meaningful information across various data sources. This paper introduces a generalizable unsupervised deep manifold learning for exploration of neurocognitive and behavioral patterns. Unlike existing methods that extract patterns directly from the input data as in the existing methods, the proposed Brain-dynamic Convolutional-Network-based Embedding (BCNE) seeks to capture the brain-state trajectories by deciphering the temporospatial correlations within the data and subsequently applying manifold learning to this correlative representation. The performance of BCNE is showcased through the analysis of several important dynamic brain datasets. The results, both visual and quantitative, reveal a diverse array of intriguing and interpretable patterns. BCNE effectively delineates scene transitions, underscores the involvement of different brain regions in memory and narrative processing, distinguishes various stages of dynamic learning processes, and identifies differences between active and passive behaviors. BCNE provides an effective tool for exploring general neuroscience inquiries or individual-specific patterns.
LLM-Based Intelligent Agents for Music Recommendation: A Comparison with Classical Content-Based Filtering
Boadana, Ronald Carvalho, Junior, Ademir Guimarรฃes da Costa, Rios, Ricardo, da Silva, Fรกbio Santos
The growing availability of music on streaming platforms has led to information overload for users. To address this issue and enhance the user experience, increasingly sophisticated recommendation systems have been proposed. This work investigates the use of Large Language Models (LLMs) from the Gemini and LLaMA families, combined with intelligent agents, in a multi-agent personalized music recommendation system. The results are compared with a traditional content-based recommendation model, considering user satisfaction, novelty, and computational efficiency. LLMs achieved satisfaction rates of up to \textit{89{,}32\%}, indicating their promising potential in music recommendation systems.
Energy-Efficient Real-Time 4-Stage Sleep Classification at 10-Second Resolution: A Comprehensive Study
Mohammadi, Zahra, Fazel, Parnian, Mohammadi, Siamak
--Sleep stage classification plays a crucial role in health monitoring, particularly for diagnosing and managing sleep disorders such as sleep apnea and insomnia. However, conventional clinical approaches like polysomnography are often costly, inconvenient, and impractical for long-term, home-based monitoring. In this study, we present an energy-efficient classification approach for detecting four sleep stages--wake, rapid eye movement (REM), light sleep, and deep sleep--using a single-lead electrocardiogram (ECG) signal. We evaluate and compare the performance of various machine-learning and deep-learning models. T o support this, we introduce two novel windowing strategies: (1) a 5-minute window with 30-second steps for machine-learning models utilizing handcrafted features, and (2) a 30-second window with 10-second steps for deep-learning models, enabling near-real-time predictions with 10-second temporal resolution. Although lightweight, our deep-learning models--such as MobileNet-v1--achieve high classification performance (up to 92% accuracy and 91% F1-score), their energy demands remain high, making them sub-optimal for wearable applications. T o address this, we design a SleepLiteCNN optimized specifically for ECG-based sleep staging. T o further enhance efficiency, we apply 8-bit quantization, which leaves classification performance unchanged, reducing the energy usage of our SleepLiteCNN to just 5.48 ยตJ per inference at a 45 nm technology node, with 90% accuracy and 90% F1-score. We further demonstrate that deploying this SleepLiteCNN on a field-programmable gate array (FPGA) significantly reduces resource usage through quantization. Overall, this approach provides a practical and efficient solution for continuous ECG-based sleep monitoring in compact, resource-constrained wearable devices. LEEP Sleep stage classification is crucial in health monitoring, especially for diagnosing and managing sleep disorders such as sleep apnea and insomnia [2]. Sleep stages are categorized into wake, REM, and Non-Rapid Eye Movement (NREM) stages. Z. Mohammadi is a Ph.D. Candidate in the School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran (e-mail: zahramo-hammmadi@ut.ac.ir). Fazel is a Graduate of the School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran (e-mail: parnian.fazel@ut.ac.ir).
Unsupervised Pairwise Learning Optimization Framework for Cross-Corpus EEG-Based Emotion Recognition Based on Prototype Representation
Li, Guangli, Wu, Canbiao, Liang, Zhen
Affective computing is a rapidly developing interdisciplinary research direction in the field of brain-computer interface. In recent years, the introduction of deep learning technology has greatly promoted the development of the field of emotion recognition. However, due to physiological differences between subjects, as well as the variations in experimental environments and equipment, cross-corpus emotion recognition faces serious challenges, especially for samples near the decision boundary. To solve the above problems, we propose an optimization method based on domain adversarial transfer learning to fine-grained alignment of affective features, named Maximum classifier discrepancy with Pairwise Learning (McdPL) framework. In McdPL, we design a dual adversarial classifier (Ada classifier and RMS classifier), and apply a three-stage adversarial training to maximize classification discrepancy and minimize feature distribution to align controversy samples near the decision boundary. In the process of domain adversarial training, the two classifiers also maintain an adversarial relationship, ultimately enabling precise cross-corpus feature alignment. In addition, the introduction of pairwise learning transforms the classification problem of samples into a similarity problem between samples, alleviating the influence of label noise. We conducted systematic experimental evaluation of the model using publicly available SEED, SEED-IV and SEED-V databases. The results show that the McdPL model is superior to other baseline models in the cross-corpus emotion recognition task, and the average accuracy improvements of 4.76\% and 3.97\%, respectively. Our work provides a promising solution for emotion recognition cross-corpus. The source code is available at https://github.com/WuCB-BCI/Mcd_PL.
Generative AI in Training and Coaching: Redefining the Design Process of Learning Materials
Komar, Alexander, Heidelmann, Marc-Andrรฉ, Schaaff, Kristina
Generative artificial intelligence (GenAI) is transforming education, redefining the role of trainers and coaches in learning environments. In our study, we explore how AI integrates into the design process of learning materials, assessing its impact on efficiency, pedagogical quality, and the evolving role of human trainers and coaches. Through qualitative interviews with professionals in education and corporate training, we identify the following key topics: trainers and coaches increasingly act as facilitators and content moderators rather than primary creators, efficiency gains allow for a stronger strategic focus but at the same time the new tools require new skills. Additionally, we analyze how the anthropomorphism of AI shapes user trust and expectations. From these insights, we derive how tools based on GenAI can successfully be implemented for trainers and coaches on an individual, organizational, systemic, and strategic level.
Sparse Attention across Multiple-context KV Cache
Cao, Ziyi, Si, Qingyi, Zhang, Jingbin, Liu, Bingquan
Large language models face significant cost challenges in long-sequence inference. To address this, reusing historical Key-Value (KV) Cache for improved inference efficiency has become a mainstream approach. Recent advances further enhance throughput by sparse attention mechanisms to select the most relevant KV Cache, thereby reducing sequence length. However, such techniques are limited to single-context scenarios, where historical KV Cache is computed sequentially with causal-attention dependencies. In retrieval-augmented generation (RAG) scenarios, where retrieved documents as context are unknown beforehand, each document's KV Cache is computed and stored independently (termed multiple-context KV Cache), lacking cross-attention between contexts. This renders existing methods ineffective. Although prior work partially recomputes multiple-context KV Cache to mitigate accuracy loss from missing cross-attention, it requires retaining all KV Cache throughout, failing to reduce memory overhead. This paper presents SamKV, the first exploration of attention sparsification for multiple-context KV Cache. Specifically, SamKV takes into account the complementary information of other contexts when sparsifying one context, and then locally recomputes the sparsified information. Experiments demonstrate that our method compresses sequence length to 15% without accuracy degradation compared with full-recompuation baselines, significantly boosting throughput in multi-context RAG scenarios.
Inductive transfer learning from regression to classification in ECG analysis
Jayasundara, Ridma, Fernando, Ishan, Fernando, Adeepa, Ragel, Roshan, Thambawita, Vajira, Nawinne, Isuru
Cardiovascular diseases (CVDs) are the leading cause of mortality worldwide, accounting for over 30% of global deaths according to the World Health Organization (WHO). Importantly, one-third of these deaths are preventable with timely and accurate diagnosis. The electrocardiogram (ECG), a non-invasive method for recording the electrical activity of the heart, is crucial for diagnosing CVDs. However, privacy concerns surrounding the use of patient ECG data in research have spurred interest in synthetic data, which preserves the statistical properties of real data without compromising patient confidentiality. This study explores the potential of synthetic ECG data for training deep learning models from regression to classification tasks and evaluates the feasibility of transfer learning to enhance classification performance on real ECG data. We experimented with popular deep learning models to predict four key cardiac parameters, namely, Heart Rate (HR), PR interval, QT interval, and QRS complex-using separate regression models. Subsequently, we leveraged these regression models for transfer learning to perform 5-class ECG signal classification. Our experiments systematically investigate whether transfer learning from regression to classification is viable, enabling better utilization of diverse open-access and synthetic ECG datasets. Our findings demonstrate that transfer learning from regression to classification improves classification performance, highlighting its potential to maximize the utility of available data and advance deep learning applications in this domain.
HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses
Deng, Zhichao, Liu, Zhikun, Wang, Junxue, Chen, Shengqian, Wei, Xiang, Yu, Qiang
Spiking Neural Networks (SNNs) offer a biologically plausible and energy-efficient framework for temporal information processing. However, existing studies overlook a fundamental property widely observed in biological neurons-synaptic heterogeneity, which plays a crucial role in temporal processing and cognitive capabilities. To bridge this gap, we introduce HetSyn, a generalized framework that models synaptic heterogeneity with synapse-specific time constants. This design shifts temporal integration from the membrane potential to the synaptic current, enabling versatile timescale integration and allowing the model to capture diverse synaptic dynamics. We implement HetSyn as HetSynLIF, an extended form of the leaky integrate-and-fire (LIF) model equipped with synapse-specific decay dynamics. By adjusting the parameter configuration, HetSynLIF can be specialized into vanilla LIF neurons, neurons with threshold adaptation, and neuron-level heterogeneous models. We demonstrate that HetSynLIF not only improves the performance of SNNs across a variety of tasks-including pattern generation, delayed match-to-sample, speech recognition, and visual recognition-but also exhibits strong robustness to noise, enhanced working memory performance, efficiency under limited neuron resources, and generalization across timescales. In addition, analysis of the learned synaptic time constants reveals trends consistent with empirical observations in biological synapses. These findings underscore the significance of synaptic heterogeneity in enabling efficient neural computation, offering new insights into brain-inspired temporal modeling.
HQ-OV3D: A High Box Quality Open-World 3D Detection Framework based on Diffision Model
Liu, Qi, Li, Yabei, Wang, Hongsong, He, Lei
Traditional closed-set 3D detection frameworks fail to meet the demands of open-world applications like autonomous driving. Existing open-vocabulary 3D detection methods typically adopt a two-stage pipeline consisting of pseudo-label generation followed by semantic alignment. While vision-language models (VLMs) recently have dramatically improved the semantic accuracy of pseudo-labels, their geometric quality, particularly bounding box precision, remains commonly neglected. To address this issue, we propose a High Box Quality Open-Vocabulary 3D Detection (HQ-OV3D) framework, dedicated to generate and refine high-quality pseudo-labels for open-vocabulary classes. The framework comprises two key components: an Intra-Modality Cross-Validated (IMCV) Proposal Generator that utilizes cross-modality geometric consistency to generate high-quality initial 3D proposals, and an Annotated-Class Assisted (ACA) Denoiser that progressively refines 3D proposals by leveraging geometric priors from annotated categories through a DDIM-based denoising mechanism. Compared to the state-of-the-art method, training with pseudo-labels generated by our approach achieves a 7.37% improvement in mAP on novel classes, demonstrating the superior quality of the pseudo-labels produced by our framework. HQ-OV3D can serve not only as a strong standalone open-vocabulary 3D detector but also as a plug-in high-quality pseudo-label generator for existing open-vocabulary detection or annotation pipelines.
Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures
Omidi, Parsa, Huang, Xingshuai, Laborieux, Axel, Nikpour, Bahareh, Shi, Tianyu, Eshaghi, Armaghan
Memory is fundamental to intelligence, enabling learning, reasoning, and adaptability across biological and artificial systems. While Transformer architectures excel at sequence modeling, they face critical limitations in long-range context retention, continual learning, and knowledge integration. This review presents a unified framework bridging neuroscience principles, including dynamic multi-timescale memory, selective attention, and consolidation, with engineering advances in Memory-Augmented Transformers. We organize recent progress through three taxonomic dimensions: functional objectives (context extension, reasoning, knowledge integration, adaptation), memory representations (parameter-encoded, state-based, explicit, hybrid), and integration mechanisms (attention fusion, gated control, associative retrieval). Our analysis of core memory operations (reading, writing, forgetting, and capacity management) reveals a shift from static caches toward adaptive, test-time learning systems. We identify persistent challenges in scalability and interference, alongside emerging solutions including hierarchical buffering and surprise-gated updates. This synthesis provides a roadmap toward cognitively-inspired, lifelong-learning Transformer architectures.