PyTorch-based Geometric Learning with Non-CUDA Processing Units: Experiences from Intel Gaudi-v2 HPUs

Graphs are a fundamental representation for a wide range of real-world systems, including social networks Nettleton [2013], molecular structures Trinajs-tic [2018], knowledge graphs Hogan et al. [2021], and recommender systems Wu et al. [2022], all of which naturally take the form of nodes connected by edges. Geometric learning, which generalizes deep learning to non-Euclidean domains Bronstein et al. [2017], especially on graph-structured data Xia et al. [2021], Wu et al. [2020], has emerged as a powerful paradigm for capturing relational and structural information. Graph neural network (GNN) models such as 1 graph convolutional networks (GCNs) Kipf and Welling [2017], graph attention networks (GATs) Veliˇ ckovi c et al. [2018], Brody et al. [2022], Lee et al. [2023], and GraphSAGE Hamilton et al. [2017] have demonstrated success across tasks from node classification Xiao et al. [2022] to link prediction Zhang and Chen [2018] and recommendation Lee et al. [2024]. While Nvidia's CUDA-enabled graphics processing units (GPUs) have become the de facto standard for accelerating deep learning workloads, there is a growing ecosystem of alternative processing units that offer different trade-offs in cost, power efficiency, and architectural specialization Lee et al. [2025]. Non-CUDA processing units, such as Google's tensor processing unit (TPUs) and Intel's Gaudi HPUs, are gaining traction in both industry and research. Supporting geometric learning workloads on these non-CUDA hardware platforms is essential to broaden hardware choice, optimize for diverse deployment scenarios, and leverage specialized hardware capabilities. Intel's Gaudi HPUs exemplify non-CUDA accelerators tailored for deep learning.