The attention mechanism was originally introduced in the context of large language models to learn relations between words [26]. Solving the many-electron Schrödinger equation for Here, the attention mechanism is employed to identify solids is an exceedingly difficult problem due to the exponential and quantify how electrons influence each other and how growth of the Hilbert space dimension. Various such influence affects their individual orbitals. This enable techniques based on the variational principle have long the construction of NN wavefunctions from Slater been developed to approximate the ground state of interacting determinants of generalized orbitals that depend on the electron systems using trial wavefunctions.

Recent neural networks demonstrated impressively accurate approximations of electronic ground-state wave functions. Such neural networks typically consist of a permutation-equivariant neural network followed by a permutation-antisymmetric operation to enforce the electronic exchange symmetry. While accurate, such neural networks are computationally expensive. In this work, we explore the flipped approach, where we first compute antisymmetric quantities based on the electronic coordinates and then apply sign equivariant neural networks to preserve the antisymmetry. While this approach promises acceleration thanks to the lower-dimensional representation, we demonstrate that it reduces to a Jastrow factor, a commonly used permutation-invariant multiplicative factor in the wave function. Our empirical results support this further, finding little to no improvements over baselines. We conclude with neither theoretical nor empirical advantages of sign equivariant functions for representing electronic wave functions within the evaluation of this work.

We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schr\"odinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.

Variational quantum Monte Carlo (QMC) is an ab-initio method for solving the electronic Schr\"odinger equation that is exact in principle, but limited by the flexibility of the available ansatzes in practice. The recently introduced deep QMC approach, specifically two deep-neural-network ansatzes PauliNet and FermiNet, allows variational QMC to reach the accuracy of diffusion QMC, but little is understood about the convergence behavior of such ansatzes. Here, we analyze how deep variational QMC approaches the fixed-node limit with increasing network size. First, we demonstrate that a deep neural network can overcome the limitations of a small basis set and reach the mean-field complete-basis-set limit. Moving to electron correlation, we then perform an extensive hyperparameter scan of a deep Jastrow factor for LiH and H$_4$ and find that variational energies at the fixed-node limit can be obtained with a sufficiently large network. Finally, we benchmark mean-field and many-body ansatzes on H$_2$O, increasing the fraction of recovered fixed-node correlation energy by half an order of magnitude compared to previous VMC results. This analysis helps understanding the superb performance of deep variational ansatzes, and will guide future improvements of the neural network architectures in deep QMC.

The electronic Schr\"odinger equation describes fundamental properties of molecules and materials, but cannot be solved exactly for larger systems than the hydrogen atom. Quantum Monte Carlo is a suitable method when high-quality approximations are sought, and its accuracy is in principle limited only by the flexibility of the used wave-function ansatz. Here we develop a deep-learning wave-function ansatz, dubbed PauliNet, which has the Hartree-Fock solution built in as a baseline, incorporates the physics of valid wave functions, and is trained using variational quantum Monte Carlo (VMC). Our deep-learning method achieves higher accuracy than comparable state-of-the-art VMC ansatzes for atoms, diatomic molecules and a strongly-correlated hydrogen chain. We anticipate that this method can reveal new physical insights and provide guidance for the design of molecules and materials where highly accurate quantum-mechanical solutions are needed, such as in transition metals and other strongly correlated systems.