All models discussed in the paper use ReLU non-linearity.

Does knowledge distillation really work?

The power of DNNs relies heavily on the quantity and quality of training data. However, collecting and annotating data on a large scale is often expensive and time-consuming.

SEEDS outperform or are competitive with previous SDE solvers.

Nevertheless, it remains elusive that whether the Transformer-based architecture can do molecular modeling as good as equivariant GNNs.

Convolutional Neural Networks (CNNs) (LeCun et al., 1989) have become a widely used and powerful tool for computer vision tasks, in large part due to their ability to achieve translation

In this paper, we specifically investigate the deep equilibrium model (DEQ), an infinite-depth neural network with shared weight matrices across layers.

We introduce a hierarchy of expressiveness classes connecting the global approximability property to the weaker property of infinite VC dimension, and prove a series of classification results for several increasingly complex functional families.