SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Models

NN, MM YYYY 1 SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Models Jinho Y ang, Graduate Student Member, IEEE, Ji-Hoon Kim, Graduate Student Member, IEEE, Joo-Y oung Kim, Senior Member, IEEE, Abstract --Deep Learning Recommendation Models (DLRMs) play a crucial role in delivering personalized content across web applications such as social networking and video streaming. However, with improvements in performance, the parameter size of DLRMs has grown to terabyte (TB) scales, accompanied by memory bandwidth demands exceeding TB/s levels. Furthermore, the workload intensity within the model varies based on the target mechanism, making it difficult to build an optimized recommendation system. In this paper, we propose SCRec, a scalable computational storage recommendation system that can handle TB-scale industrial DLRMs while guaranteeing high bandwidth requirements. SCRec utilizes a software framework that features a mixed-integer programming (MIP)-based cost model, efficiently fetching data based on data access patterns and adaptively configuring memory-centric and compute-centric cores. Additionally, SCRec integrates hardware acceleration cores to enhance DLRM computations, particularly allowing for the high-performance reconstruction of approximated embedding vectors from extremely compressed tensor-train (TT) format. By combining its software framework and hardware accelerators, while eliminating data communication overhead by being implemented on a single server, SCRec achieves substantial improvements in DLRM inference performance. It delivers up to 55.77 speedup compared to a CPU-DRAM system with no loss in accuracy and up to 13.35 energy efficiency gains over a multi-GPU system. I NTRODUCTION R RECOMMENDA TION systems are widely used in social network services and video streaming platforms to provide personalized and preferred content to consumers as described in Fig.1. They are also employed in search engines to offer differentiated search services [1]-[5]. For example, more than 80% of Meta's data center resources are allocated to recommendation system inference, while over 50% are utilized for training these systems [6]. Traditional recommendation systems relied on collaborative filtering techniques, such as content filtering using matrix factorization [7]-[10]. However, with advancements in deep neural networks (DNNs), deep learning recommendation models (DLRMs) that combine embedding tables (EMBs) and This work was supported by Samsung Electronics Co., Ltd.. Manuscript received MM dd, yyyy; revised MM dd, yyyy. These models are widely adopted in data centers, with recent focuses on both software-level and hardware-level optimizations [11]- [17]. This combination has demonstrated superior recommendation performance, making DLRM the industry standard in recommendation systems.