Goto

Collaborating Authors

 stca


Make It Long, Keep It Fast: End-to-End 10k-Sequence Modeling at Billion Scale on Douyin

arXiv.org Artificial Intelligence

Short-video recommenders such as Douyin must exploit extremely long user histories without breaking latency or cost budgets. We present an end-to-end system that scales long-sequence modeling to 10k-length histories in production. First, we introduce Stacked Target-to-History Cross Attention (STCA), which replaces history self-attention with stacked cross-attention from the target to the history, reducing complexity from quadratic to linear in sequence length and enabling efficient end-to-end training. Second, we propose Request Level Batching (RLB), a user-centric batching scheme that aggregates multiple targets for the same user/request to share the user-side encoding, substantially lowering sequence-related storage, communication, and compute without changing the learning objective. Third, we design a length-extrapolative training strategy -- train on shorter windows, infer on much longer ones -- so the model generalizes to 10k histories without additional training cost. Across offline and online experiments, we observe predictable, monotonic gains as we scale history length and model capacity, mirroring the scaling law behavior observed in large language models. Deployed at full traffic on Douyin, our system delivers significant improvements on key engagement metrics while meeting production latency, demonstrating a practical path to scaling end-to-end long-sequence recommendation to the 10k regime.


An indicator for effectiveness of text-to-image guardrails utilizing the Single-Turn Crescendo Attack (STCA)

arXiv.org Artificial Intelligence

The Single-Turn Crescendo Attack (STCA), first introduced in Aqrawi and Abbasi [2024], is an innovative method designed to bypass the ethical safeguards of text-to-text AI models, compelling them to generate harmful content. This technique leverages a strategic escalation of context within a single prompt, combined with trust-building mechanisms, to subtly deceive the model into producing unintended outputs. Extending the application of STCA to text-to-image models, we demonstrate its efficacy by compromising the guardrails of a widely-used model, DALL-E 3, achieving outputs comparable to outputs from the uncensored model Flux Schnell, which served as a baseline control. This study provides a framework for researchers to rigorously evaluate the robustness of guardrails in text-to-image models and benchmark their resilience against adversarial attacks.


Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA)

arXiv.org Artificial Intelligence

This paper introduces a new method for adversarial attacks on large language models (LLMs) called the Single-Turn Crescendo Attack (STCA). Building on the multi-turn crescendo attack method introduced by Russinovich, Salem, and Eldan (2024), which gradually escalates the context to provoke harmful responses, the STCA achieves similar outcomes in a single interaction. By condensing the escalation into a single, well-crafted prompt, the STCA bypasses typical moderation filters that LLMs use to prevent inappropriate outputs. This technique reveals vulnerabilities in current LLMs and emphasizes the importance of stronger safeguards in responsible AI (RAI). The STCA offers a novel method that has not been previously explored. Figure 1: A direct prompt asking the model to create sentences with the word "fuck" or generate a hypothetical scenario where a world-renowned musician is related to Hitler, would result in the model rejecting the request outright. However, the STCA enables a more condensed, escalating structure, allowing the model to gradually accept the input and continue the dialogue while bypassing its typical moderation filters. For the above examples, we used just 3 turns within a single STCA prompt to achieve this, referred to as an STCA-3.


Spatial-Temporal Convolutional Attention for Mapping Functional Brain Networks

arXiv.org Machine Learning

Recently, to overcome the shallow nature of the linear models, various of deep learning based methods have been Using functional magnetic resonance imaging (fMRI) and proposed to discover the FBNs. Most of these methods are deep learning to explore functional brain networks (FBNs) based on the autoencoders, they use different autoencoders has attracted many researchers. However, most of these to extract the sources in an self-supervised manner, and then studies are still based on the temporal correlation between use the generative linear model, such as LASSO to generate the sources and voxel signals, and lack of researches on the the FBNs [6, 7]. In general, these deep learning based methods dynamics of brain function. Due to the widespread local can indeed extract better encoder representations as the correlations in the volumes, FBNs can be generated directly sources than the classical methods, such as ICA and SDL, but in the spatial domain in a self-supervised manner by using still generate FBNs in a linear and independent manner, with spatial-wise attention (SA), and the resulting FBNs has the sources extraction and the FBNs generation as 2 separate a higher spatial similarity with templates compared to the steps. Generating the FBNs in such way is time-consuming classical method. Therefore, we proposed a novel Spatial-and does not fully utilize the advantages of deep learning, and Temporal Convolutional Attention (STCA) model to discover cannot directly generate the FBNs with deep learning.


Reconstruction of Privacy-Sensitive Data from Protected Templates

arXiv.org Machine Learning

In this paper, we address the problem of data reconstruction from privacy-protected templates, based on recent concept of sparse ternary coding with ambiguization (STCA). The STCA is a generalization of randomization techniques which includes random projections, lossy quantization, and addition of ambiguization noise to satisfy the privacy-utility trade-off requirements. The theoretical privacy-preserving properties of STCA have been validated on synthetic data. However, the applicability of STCA to real data and potential threats linked to reconstruction based on recent deep reconstruction algorithms are still open problems. Our results demonstrate that STCA still achieves the claimed theoretical performance when facing deep reconstruction attacks for the synthetic i.i.d. data, while for real images special measures are required to guarantee proper protection of the templates.