Complete and Efficient Graph Transformers for Crystal Material Property Prediction

Crystal structures are characterized by atomic bases within a primitive unit cell that repeats along a regular lattice throughout 3D space. The periodic and infinite nature of crystals poses unique challenges for geometric graph representation learning. Specifically, constructing graphs that effectively capture the complete geometric information of crystals and handle chiral crystals remains an unsolved and challenging problem. In this paper, we introduce a novel approach that utilizes the periodic patterns of unit cells to establish the lattice-based representation for each atom, enabling efficient and expressive graph representations of crystals. Furthermore, we propose ComFormer, a SE(3) transformer designed specifically for crystalline materials. ComFormer includes two variants; namely, iComFormer that employs invariant geometric descriptors of Euclidean distances and angles, and eComFormer that utilizes equivariant vector representations. Experimental results demonstrate the state-of-the-art predictive accuracy of ComFormer variants on various tasks across three widely-used crystal benchmarks. Our code is publicly available as part of the AIRS library (https://github.com/divelab/AIRS). However, the current reliance on traditional, costly, and time-consuming trial-and-error experimental methods poses practical challenges. In this regard, computational approaches based on quantum mechanics, such as density functional theory (DFT), have made significant contributions for predicting the physical and chemical properties of materials to guide materials discovery experiments. However, these crystal graph representations have limitations in distinguishing different crystalline materials. In other words, they cannot guarantee to capture the complete geometric information of input crystal structures, and may map different crystal structures with different properties to the same graph representation and produce the identical property predictions. Illustrative examples and detailed discussions can be found in Appendix A.1. While graph representations that can capture any structural differences in small molecules have been investigated in previous works (Wang et al., 2022; Klicpera et al., 2021), these methods fail short to capture periodic patterns of crystals and cannot maintain geometric completeness for crystals.