We propose a new representation for encoding 3D shapes as neural fields. The representation isdesignedtobecompatible withthetransformer architecture and to benefit both shape reconstruction and shape generation. Existing works on neural fields aregrid-based representations withlatents defined onaregular grid.

In graph learning domain, inspired by the breakthroughs, multiple works tried combining selfattention into graph neural network (GNN) architecture where message passing was previously dominant[50].

Using theneural tangent kernel (NTK), we demonstrate that this phenomena occurs inpart because the NTK is not shift-invariant.