sra
Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection
Eilertsen, Brage, Bjørgfinsdóttir, Røskva, Vargas, Francielle, Ramezani-Kebrya, Ali
The opaque nature of deep learning models presents significant challenges for the ethical deployment of hate speech detection systems. To address this limitation, we introduce Supervised Rational Attention (SRA), a framework that explicitly aligns model attention with human rationales, improving both interpretability and fairness in hate speech classification. SRA integrates a supervised attention mechanism into transformer-based classifiers, optimizing a joint objective that combines standard classification loss with an alignment loss term that minimizes the discrepancy between attention weights and human-annotated rationales. We evaluated SRA on hate speech benchmarks in English (HateXplain) and Portuguese (HateBRXplain) with rationale annotations. Empirically, SRA achieves 2.4x better explainability compared to current baselines, and produces token-level explanations that are more faithful and human-aligned. In terms of fairness, SRA achieves competitive fairness across all measures, with second-best performance in detecting toxic posts targeting identity groups, while maintaining comparable results on other metrics. These findings demonstrate that incorporating human rationales into attention mechanisms can enhance interpretability and faithfulness without compromising fairness.
- Asia > Middle East > UAE > Abu Dhabi Emirate > Abu Dhabi (0.14)
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- North America > United States > California > San Francisco County > San Francisco (0.14)
- (14 more...)
- Law (0.93)
- Information Technology > Security & Privacy (0.93)
Opening the Black Box: Interpretable LLMs via Semantic Resonance Architecture
Large language models (LLMs) achieve remarkable performance but remain difficult to interpret. Mixture-of-Experts (MoE) models improve efficiency through sparse activation, yet typically rely on opaque, learned gating functions. While similarity-based routing (Cosine Routers) has been explored for training stabilization, its potential for inherent interpretability remains largely untapped. We introduce the Semantic Resonance Architecture (SRA), an MoE approach designed to ensure that routing decisions are inherently interpretable. SRA replaces learned gating with a Chamber of Semantic Resonance (CSR) module, which routes tokens based on cosine similarity with trainable semantic anchors. We also introduce a novel Dispersion Loss that encourages orthogonality among anchors to enforce diverse specialization. Experiments on WikiText-103 demonstrate that SRA achieves a validation perplexity of 13.41, outperforming both a dense baseline (14.13) and a Standard MoE baseline (13.53) under matched active parameter constraints (29.0M). Crucially, SRA exhibits superior expert utilization (1.0% dead experts vs. 14.8% in the Standard MoE) and develops distinct, semantically coherent specialization patterns, unlike the noisy specialization observed in standard MoEs. This work establishes semantic routing as a robust methodology for building more transparent and controllable language models.
- Europe > Ukraine (0.04)
- Asia > Middle East > Jordan (0.04)
Exploring Selective Retrieval-Augmentation for Long-Tail Legal Text Classification
Legal text classification is a fundamental NLP task in the legal domain. Benchmark datasets in this area often exhibit a long-tail label distribution, where many labels are underrepresented, leading to poor model performance on rare classes. This paper explores Selective Retrieval-Augmentation (SRA) as a proof-of-concept approach to this problem. SRA focuses on augmenting samples belonging to low-frequency labels in the training set, preventing the introduction of noise for well-represented classes, and requires no changes to the model architecture. Retrieval is performed only from the training data to ensure there is no potential information leakage, removing the need for external corpora simultaneously. SRA is tested on two legal text classification benchmark datasets with long-tail distributions: LEDGAR (single-label) and UNFAIR-ToS (multi-label). Results show that SRA achieves consistent gains in both micro-F1 and macro-F1 over LexGLUE baselines.
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.14)
- Europe > Ireland > Leinster > County Dublin > Dublin (0.04)
- Europe > France > Provence-Alpes-Côte d'Azur > Bouches-du-Rhône > Marseille (0.04)
Structured Regularization for Constrained Optimization on the SPD Manifold
Matrix-valued optimization tasks, including those involving symmetric positive definite (SPD) matrices, arise in a wide range of applications in machine learning, data science and statistics. Classically, such problems are solved via constrained Euclidean optimization, where the domain is viewed as a Euclidean space and the structure of the matrices (e.g., positive definiteness) enters as constraints. More recently, geometric approaches that leverage parametrizations of the problem as unconstrained tasks on the corresponding matrix manifold have been proposed. While they exhibit algorithmic benefits in many settings, they cannot directly handle additional constraints, such as inequality or sparsity constraints. A remedy comes in the form of constrained Riemannian optimization methods, notably, Riemannian Frank-Wolfe and Projected Gradient Descent. However, both algorithms require potentially expensive subroutines that can introduce computational bottlenecks in practise. To mitigate these shortcomings, we introduce a class of structured regularizers, based on symmetric gauge functions, which allow for solving constrained optimization on the SPD manifold with faster unconstrained methods. We show that our structured regularizers can be chosen to preserve or induce desirable structure, in particular convexity and "difference of convex" structure. We demonstrate the effectiveness of our approach in numerical experiments.
- North America > United States > New Jersey > Mercer County > Princeton (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Europe > Germany > North Rhine-Westphalia > Upper Bavaria > Munich (0.04)
Steering Conversational Large Language Models for Long Emotional Support Conversations
Madani, Navid, Saha, Sougata, Srihari, Rohini
In this study, we address the challenge of consistently following emotional support strategies in long conversations by large language models (LLMs). We introduce the Strategy-Relevant Attention (SRA) metric, a model-agnostic measure designed to evaluate the effectiveness of LLMs in adhering to strategic prompts in emotional support contexts. By analyzing conversations within the Emotional Support Conversations dataset (ESConv) using LLaMA models, we demonstrate that SRA is significantly correlated with a model's ability to sustain the outlined strategy throughout the interactions. Our findings reveal that the application of SRA-informed prompts leads to enhanced strategic adherence, resulting in conversations that more reliably exhibit the desired emotional support strategies over longer conversations. Furthermore, we contribute a comprehensive, multi-branch synthetic conversation dataset for ESConv, featuring a variety of strategy continuations informed by our optimized prompting method. The code and data are publicly available on our Github.
- Health & Medicine > Consumer Health (0.93)
- Health & Medicine > Therapeutic Area > Psychiatry/Psychology > Mental Health (0.68)
A Human Motion Compensation Framework for a Supernumerary Robotic Arm
Zhang, Xin, Balatti, Pietro, Leonori, Mattia, Ajoudani, Arash
Supernumerary robotic arms (SRAs) can be used as the third arm to complement and augment the abilities of human users. The user carrying a SRA forms a connected kinodynamic chain, which can be viewed as a special class of floating-base robot systems. However, unlike general floating-base robot systems, human users are the bases of SRAs and they have their subjective behaviors/motions. This implies that human body motions can unintentionally affect the SRA's end-effector movements. To address this challenge, we propose a framework to compensate for the human whole-body motions that interfere with the SRA's end-effector trajectories. The SRA system in this study consists of a 6-degree-of-freedom lightweight arm and a wearable interface. The wearable interface allows users to adjust the installation position of the SRA to fit different body shapes. An inertial measurement unit (IMU)-based sensory interface can provide the body skeleton motion feedback of the human user in real time. By simplifying the floating-base kinematics model, we design an effective motion planner by reconstructing the Jacobian matrix of the SRA. Under the proposed framework, the performance of the reconstructed Jacobian method is assessed by comparing the results obtained with the classical nullspace-based method through two sets of experiments.
Manager/ Principal Scientist - Speech ML Scientist at Samsung Research America - Mountain View, CA
Overview: Samsung Research America (SRA) plays a pivotal role in developing the next generation of discovery in software, user experience and services for future products that can enrich your life. Our mission is to research and develop new technologies by partnering with the best and brightest and creating a collaborative environment between industry and academia. Headquartered in Silicon Valley, with locations in many technology centers in North America, SRA is driven to build a culture of innovation that rapidly translates research and new ideas into the unexpected. Bixby is an intelligent personal assistant which is only available as a built-in service on Samsung flagship devices and wearables. Bixby uses state-of-art Speech Recognition & Natural Language Processing and Knowledge-Based AI to perform tasks on these devices using multimodal inputs and additional contextual information, including but not limited to making phone calls, sending text messages, setting up meetings, opening apps, setting alarms and timers, getting directions, answering general questions, providing information about restaurants and other businesses, etc.
- Semiconductors & Electronics (0.94)
- Law (0.77)
Senior Staff/Principal Scientist - Speech ML Scientist at Samsung Research America - Mountain View, CA
Overview: Samsung Research America (SRA) plays a pivotal role in developing the next generation of discovery in software, user experience and services for future products that can enrich your life. Our mission is to research and develop new technologies by partnering with the best and brightest and creating a collaborative environment between industry and academia. Headquartered in Silicon Valley, with locations in many technology centers in North America, SRA is driven to build a culture of innovation that rapidly translates research and new ideas into the unexpected. We have the power to enrich lives. And that's what making a better global society is all about.
Machine Learning Research Engineer
Overview: Samsung Research America (SRA) plays a pivotal role in developing the next generation of discovery in software, user experience and services for future products that can enrich your life. Our mission is to research and develop new technologies by partnering with the best and brightest and creating a collaborative environment between industry and academia. Headquartered in Silicon Valley, with locations in many technology centers in North America, SRA is driven to build a culture of innovation that rapidly translates research and new ideas into the unexpected. We have the power to enrich lives. And that's what making a better global society is all about.
Senior Research Scientist - On-Device Machine Learning
For U.S. Candidates Only: SRA has adopted a COVID-19 vaccination policy to safeguard the health and well-being of our employees and visitors. As a condition of employment, all employees based in the U.S. are required to be fully vaccinated for COVID-19, unless a reasonable accommodation is approved or as otherwise required by law. Incumbent must make themselves available during core business hours. This position requires the incumbent to travel for work 10% of the time.