Molecular Quantum Transformer

Molecular Quantum Transformer Yuichi Kamata, Quoc Hoan Tran, Yasuhiro Endo, and Hirotaka Oshima Quantum Laboratory, Fujitsu Research, Fujitsu Limited, Kawasaki, Kanagawa 211-8588, Japan The Transformer model, renowned for its powerful attention mechanism, has achieved state-of-the-art performance in various artificial intelligence tasks but faces challenges such as high computational cost and memory usage. Researchers are exploring quantum computing to enhance the Transformer's design, though it still shows limited success with classical data. With a growing focus on leveraging quantum machine learning for quantum data, particularly in quantum chemistry, we propose the Molecular Quantum Transformer (MQT) for modeling interactions in molecular quantum systems. By utilizing quantum circuits to implement the attention mechanism on the molecular configurations, MQT can efficiently calculate ground-state energies for all configurations. Numerical demonstrations show that in calculating ground-state energies for H 2, LiH, BeH 2, and H 4, MQT outperforms the classical Transformer, highlighting the promise of quantum effects in Transformer structures. Our method offers an alternative to existing quantum algorithms for estimating ground-state energies, opening new avenues in quantum chemistry and materials science. I. INTRODUCTION The Transformer model [1] has been recognized as a remarkable advancement in artificial intelligence. Its key power lies in its "attention mechanism", which discerns the relative importance of different parts of its input and the connection strengths between them. This mechanism has been successfully applied to both natural language processing and visual object recognition tasks, delivering state-of-the-art performance across a variety of datasets. Despite these successes, the current implementation of the Transformer faces several challenges, including high computational costs, substantial memory requirements, the necessity for large datasets, and a vast number of training parameters. These limitations have prompted researchers to explore improved Transformer designs.