HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration

Subsequently, the feed-forward Transformers have revolutionized deep learning and generative (FF) network is employed, which includes multiplication with the modeling to enable unprecedented advancements in natural trainable weights. The end-to-end transformer model also consists language processing tasks and beyond. However, designing of additional computations such as softmax, layer-normalization, hardware accelerators for executing transformer models is activation function, positional encoding, etc. These computational challenging due to the wide variety of computing kernels involved kernels give rise to the heterogeneity of operations in the in the transformer architecture. Existing accelerators are either transformer architecture. Recently, processing-in-memory (PIM) inadequate to accelerate end-to-end transformer models or suffer has emerged as a promising approach to accelerate the notable thermal limitations. In this paper, we propose the design of training/inference of deep neural networks (DNNs) [2]. Emerging a three-dimensional heterogeneous architecture referred to as resistive random-access memory (ReRAM)-based PIM HeTraX specifically optimized to accelerate end-to-end architectures can achieve higher performance and better energy transformer models. HeTraX employs hardware resources aligned efficiency than GPU-based counterparts [2].