speeder
A Remarkably Efficient Paradigm to Multimodal Large Language Models for Sequential Recommendation
Zhong, Qiyong, Su, Jiajie, Yang, Ming, Ma, Yunshan, Zheng, Xiaolin, Chen, Chaochao
Sequential recommendations (SR) predict users' future interactions based on their historical behavior. The rise of Large Language Models (LLMs) has brought powerful generative and reasoning capabilities, significantly enhancing SR performance, while Multimodal LLMs (MLLMs) further extend this by introducing data like images and interactive relationships. However, critical issues remain, i.e., (a) Suboptimal item representations caused by lengthy and redundant descriptions, leading to inefficiencies in both training and inference; (b) Modality-related cognitive bias, as LLMs are predominantly pretrained on textual data, limiting their ability to effectively integrate and utilize non-textual modalities; (c) Weakening sequential perception in long interaction sequences, where attention mechanisms struggle to capture earlier interactions, hindering the modeling of long-range dependencies. To address these issues, we propose Speeder, an efficient MLLM-based paradigm for SR featuring three key innovations: 1) Multimodal Representation Compression (MRC), which condenses item attributes into concise yet informative tokens, reducing redundancy and computational cost; 2) Modality-aware Progressive Optimization (MPO), enabling gradual learning of multimodal representations; 3) Sequential Position Awareness Enhancement (SPAE), improving the LLM's capability to capture both relative and absolute sequential dependencies in long interaction sequences. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of Speeder. Speeder increases training speed to 250% of the original while reducing inference time to 25% on the Amazon dataset.
- North America > United States > District of Columbia > Washington (0.05)
- Asia > China > Zhejiang Province > Hangzhou (0.04)
- Asia > Singapore (0.04)
- (2 more...)
How a jetpack design helped create a flying motorbike
At around the age of 12, David Mayman tried to build a helicopter out of fence posts and an old lawn mower. Needless to say, it did not go well. His contraption didn't fly and he was made to fix the fence. "I was brought up in a way that I guess challenged me scientifically... I was always told that nothing's impossible," he says.
- North America > United States > California > Los Angeles County > Los Angeles (0.05)
- Asia > Taiwan (0.05)
- Asia > India (0.05)
- Transportation > Air (1.00)
- Aerospace & Defense (1.00)
- Energy (0.97)