Large Language Model
SugarTextNet: A Transformer-Based Framework for Detecting Sugar Dating-Related Content on Social Media with Context-Aware Focal Loss
Wang, Lionel Z., Ben, Shihan, Huang, Yulu, Qin, Simeng
Sugar dating-related content has rapidly proliferated on mainstream social media platforms, giving rise to serious societal and regulatory concerns, including commercialization of intimate relationships and the normalization of transactional relationships. Detecting such content is highly challenging due to the prevalence of subtle euphemisms, ambiguous linguistic cues, and extreme class imbalance in real-world data. In this work, we present SugarT extNet, a novel transformer-based framework specifically designed to identify sugar dating-related posts on social media. SugarT extNet integrates a pretrained transformer encoder, an attention-based cue extractor, and a contextual phrase encoder to capture both salient and nuanced features in user-generated text. T o address class imbalance and enhance minority-class detection, we introduce Context-Aware F ocal Loss, a tailored loss function that combines focal loss scaling with contextual weighting. W e evaluate SugarT extNet on a newly curated, manually annotated dataset of 3,067 Chinese social media posts from Sina W eibo, demonstrating that our approach substantially outperforms traditional machine learning models, deep learning baselines, and large language models across multiple metrics. Comprehensive ablation studies confirm the indispensable role of each component. Our findings highlight the importance of domain-specific, context-aware modeling for sensitive content detection, and provide a robust solution for content moderation in complex, real-world scenarios.
LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation
Shi, Teng, Shen, Chenglei, Yu, Weijie, Nie, Shen, Li, Chongxuan, Zhang, Xiao, He, Ming, Han, Yan, Xu, Jun
Generative recommendation represents each item as a semantic ID, i.e., a sequence of discrete tokens, and generates the next item through autoregressive decoding. While effective, existing autoregressive models face two intrinsic limitations: (1) unidirectional constraints, where causal attention restricts each token to attend only to its predecessors, hindering global semantic modeling; and (2) error accumulation, where the fixed left-to-right generation order causes prediction errors in early tokens to propagate to the predictions of subsequent token. To address these issues, we propose LLaDA-Rec, a discrete diffusion framework that reformulates recommendation as parallel semantic ID generation. By combining bidirectional attention with the adaptive generation order, the approach models inter-item and intra-item dependencies more effectively and alleviates error accumulation. Specifically, our approach comprises three key designs: (1) a parallel tokenization scheme that produces semantic IDs for bidirectional modeling, addressing the mismatch between residual quantization and bidirectional architectures; (2) two masking mechanisms at the user-history and next-item levels to capture both inter-item sequential dependencies and intra-item semantic relationships; and (3) an adapted beam search strategy for adaptive-order discrete diffusion decoding, resolving the incompatibility of standard beam search with diffusion-based generation. Experiments on three real-world datasets show that LLaDA-Rec consistently outperforms both ID-based and state-of-the-art generative recommenders, establishing discrete diffusion as a new paradigm for generative recommendation.
SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
Ma, Jeffrey Jian, Hashemi, Milad, Yazdanbakhsh, Amir, Swersky, Kevin, Press, Ofir, Li, Enhui, Reddi, Vijay Janapa, Ranganathan, Parthasarathy
Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correctness. However, most benchmarks emphasize what to fix rather than how to fix code. We introduce SWE-fficiency, a benchmark for evaluating repository-level performance optimization on real workloads. Our suite contains 498 tasks across nine widely used data-science, machine-learning, and HPC repositories (e.g., numpy, pandas, scipy): given a complete codebase and a slow workload, an agent must investigate code semantics, localize bottlenecks and relevant tests, and produce a patch that matches or exceeds expert speedup while passing the same unit tests. To enable this how-to-fix evaluation, our automated pipeline scrapes GitHub pull requests for performance-improving edits, combining keyword filtering, static analysis, coverage tooling, and execution validation to both confirm expert speedup baselines and identify relevant repository unit tests. Empirical evaluation of state-of-the-art agents reveals significant underperformance. On average, agents achieve less than 0.15x the expert speedup: agents struggle in localizing optimization opportunities, reasoning about execution across functions, and maintaining correctness in proposed edits. We release the benchmark and accompanying data pipeline to facilitate research on automated performance engineering and long-horizon software reasoning.
A Remarkably Efficient Paradigm to Multimodal Large Language Models for Sequential Recommendation
Zhong, Qiyong, Su, Jiajie, Yang, Ming, Ma, Yunshan, Zheng, Xiaolin, Chen, Chaochao
Sequential recommendations (SR) predict users' future interactions based on their historical behavior. The rise of Large Language Models (LLMs) has brought powerful generative and reasoning capabilities, significantly enhancing SR performance, while Multimodal LLMs (MLLMs) further extend this by introducing data like images and interactive relationships. However, critical issues remain, i.e., (a) Suboptimal item representations caused by lengthy and redundant descriptions, leading to inefficiencies in both training and inference; (b) Modality-related cognitive bias, as LLMs are predominantly pretrained on textual data, limiting their ability to effectively integrate and utilize non-textual modalities; (c) Weakening sequential perception in long interaction sequences, where attention mechanisms struggle to capture earlier interactions, hindering the modeling of long-range dependencies. To address these issues, we propose Speeder, an efficient MLLM-based paradigm for SR featuring three key innovations: 1) Multimodal Representation Compression (MRC), which condenses item attributes into concise yet informative tokens, reducing redundancy and computational cost; 2) Modality-aware Progressive Optimization (MPO), enabling gradual learning of multimodal representations; 3) Sequential Position Awareness Enhancement (SPAE), improving the LLM's capability to capture both relative and absolute sequential dependencies in long interaction sequences. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of Speeder. Speeder increases training speed to 250% of the original while reducing inference time to 25% on the Amazon dataset.
DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning
Wang, Yaxuan, Liu, Chris Yuhao, Liu, Quan, Pang, Jinglong, Wei, Wei, Bao, Yujia, Liu, Yang
Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose Detect-Reasoning Augmented GeneratiON (DRAGON), a systematic, reasoning-based framework that utilizes in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.
SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
Xu, Jingxuan, Deng, Ken, Li, Weihao, Yu, Songwei, Tang, Huaixi, Huang, Haoyang, Lai, Zhiyi, Zhan, Zizheng, Wu, Yanan, Zhang, Chenchen, Lei, Kepeng, Yao, Yifan, Lei, Xinping, Zhu, Wenqiang, Feng, Zongxian, Li, Han, Xiong, Junqi, Li, Dailin, Gao, Zuchen, Wu, Kun, Xiang, Wen, Zhan, Ziqi, Zhang, Yuanxing, Gong, Wuxuan, Gao, Ziyuan, Wang, Guanxiang, Xue, Yirong, Li, Mengtong, Xie, Mengfei, Zhang, Xiaojiang, Wang, Jinghui, Zhuang, Wenhao, Lin, Zheng, Wang, Huiming, Zhang, Zhaoxiang, Zhang, Yuqun, Zhang, Haotian, Chen, Bin, Liu, Jiaheng
Evaluating large language models (LLMs) for software engineering has been limited by narrow task coverage, language bias, and insufficient alignment with real-world developer workflows. Existing benchmarks often focus on algorithmic problems or Python-centric bug fixing, leaving critical dimensions of software engineering underexplored. To address these gaps, we introduce SWE-Compass1, a comprehensive benchmark that unifies heterogeneous code-related evaluations into a structured and production-aligned framework. SWE-Compass spans 8 task types, 8 programming scenarios, and 10 programming languages, with 2000 high-quality instances curated from authentic GitHub pull requests and refined through systematic filtering and validation. We benchmark ten state-of-the-art LLMs under two agentic frameworks, SWE-Agent and Claude Code, revealing a clear hierarchy of difficulty across task types, languages, and scenarios. Moreover, by aligning evaluation with real-world developer practices, SWE-Compass provides a rigorous and reproducible foundation for diagnosing and advancing agentic coding capabilities in large language models.
OMPILOT: Harnessing Transformer Models for Auto Parallelization to Shared Memory Computing Paradigms
Bhattacharjee, Arijit, TehraniJamsaz, Ali, Chen, Le, Hasabnis, Niranjan, Capota, Mihai, Ahmed, Nesreen, Jannesari, Ali
Recent advances in large language models (LLMs) have significantly accelerated progress in code translation, enabling more accurate and efficient transformation across programming languages. While originally developed for natural language processing, LLMs have shown strong capabilities in modeling programming language syntax and semantics, outperforming traditional rule-based systems in both accuracy and flexibility. These models have streamlined cross-language conversion, reduced development overhead, and accelerated legacy code migration. In this paper, we introduce OMPILOT, a novel domain-specific encoder-decoder transformer tailored for translating C++ code into OpenMP, enabling effective shared-memory parallelization. OMPILOT leverages custom pre-training objectives that incorporate the semantics of parallel constructs and combines both unsupervised and supervised learning strategies to improve code translation robustness. Unlike previous work that focused primarily on loop-level transformations, OMPILOT operates at the function level to capture a wider semantic context. To evaluate our approach, we propose OMPBLEU, a novel composite metric specifically crafted to assess the correctness and quality of OpenMP parallel constructs, addressing limitations in conventional translation metrics.
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
Zhou, Shijie, Lai, Viet Dac, Tan, Hao, Kil, Jihyung, Zhu, Wanrong, Chen, Changyou, Zhang, Ruiyi
Specifically, GUI-AIMA-3B is better than strong large size coordinate-based UI-T ARS-1.5-7B, JEDI-7B, and also better than the embedding-based coordinate-free GUI-Actor-7B model, highlighting the superiority of directly supervising on the multi-head self-attention weights instead of modeling the query-visual attention map via hidden states. On ScreenSpot-v2, GUI-AIMA-3B achieve the comparable results with the strongest baselines, such as JEDI-7B and UI-T ARS-7B, and better than the same size GUI-Actor-3B, while GUI-AIMA-3B is trained with much less web data. Besides the advanced performance, another advantage is the training efficiency of GUI-AIMA with only 259k training elements, as most supervised fine-tuned baselines trained on millions of GUI elements. For the comparison with reinforcement fine-tuned baselines, while both trained with smaller training sets than SFT baselines, GUI-AIMA-3B performs better and shows better generalization. Among GUI-AIMA-3B and GUI-AIMA-3B (soft), GUI-AIMA-3B is slightly better on dealing with diverse graphic environments in ScreenSpot-v2, OSWorld-G and ScreenSpot-pro. The two-step inference with zoom-in without extra training significantly improves the performance of GUI-AIMA on high-resolution benchmarks, ScreenSpot-pro and OSWorld-G, specifically 59.6% on ScreenSpot-pro using GUI-AIMA-3B (soft) and 63.8% on OSWorld-G using GUI-AIMA-3B, demonstrating the flexibility of GUI-AIMA's attention-based patch-wise grounding for inference-time improvements.
EdgeRunner 20B: Military Task Parity with GPT-5 while Running on the Edge
FitzGerald, Jack, Lazaridis, Aristotelis, Bates, Dylan, Sharma, Aman, Castillo, Jonnathan, Azami, Yousif, Bailey, Sean, Cao, Jeremy, Damianov, Peter, de Haan, Kevin, Kerbs, Luke, Lu, Vincent, Madigan, Joseph, McLaurin, Jeremy, Tainer, Jonathan, Anderson, Dave, Beck, Jonathan, Cuticello, Jamie, Malkerson, Colton, Saltsman, Tyler
We present EdgeRunner 20B, a fine-tuned version of gpt-oss-20b optimized for military tasks. EdgeRunner 20B was trained on 1.6M high-quality records curated from military documentation and websites. We also present four new tests sets: (a) combat arms, (b) combat medic, (c) cyber operations, and (d) mil-bench-5k (general military knowledge). On these military test sets, EdgeRunner 20B matches or exceeds GPT-5 task performance with 95%+ statistical significance, except for the high reasoning setting on the combat medic test set and the low reasoning setting on the mil-bench-5k test set. Versus gpt-oss-20b, there is no statistically-significant regression on general-purpose benchmarks like ARC-C, GPQA Diamond, GSM8k, IFEval, MMLU Pro, or TruthfulQA, except for GSM8k in the low reasoning setting. We also present analyses on hyperparameter settings, cost, and throughput. These findings show that small, locally-hosted models are ideal solutions for data-sensitive operations such as in the military domain, allowing for deployment in air-gapped edge devices.
Chain-of-Thought Hijacking
Zhao, Jianli, Fu, Tingchen, Schaeffer, Rylan, Sharma, Mrinank, Barez, Fazl
Large reasoning models (LRMs) achieve higher task performance with more inference-time computation, and prior works suggest this scaled reasoning may also strengthen safety by improving refusal. Yet we find the opposite: the same reasoning can be used to bypass safeguards. We introduce Chain-of-Thought Hijacking, a jailbreak attack on reasoning models. The attack pads harmful requests with long sequences of harmless puzzle reasoning. Across HarmBench, CoT Hijacking reaches a 99%, 94%, 100%, and 94% attack success rate (ASR) on Gemini 2.5 Pro, GPT o4 mini, Grok 3 mini, and Claude 4 Sonnet, respectively - far exceeding prior jailbreak methods for LRMs. To understand the effectiveness of our attack, we turn to a mechanistic analysis, which shows that mid layers encode the strength of safety checking, while late layers encode the verification outcome. Long benign CoT dilutes both signals by shifting attention away from harmful tokens. Targeted ablations of attention heads identified by this analysis causally decrease refusal, confirming their role in a safety subnetwork. These results show that the most interpretable form of reasoning - explicit CoT - can itself become a jailbreak vector when combined with final-answer cues. We release prompts, outputs, and judge decisions to facilitate replication.