Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models

In this paper, we specifically investigate the deep equilibrium model (DEQ), an infinite-depth neural network with shared weight matrices across layers.