Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks

For deep linear networks, these kernels satisfy a set of algebraic matrix equations. For nonlinear networks, we provide an alternating sampling procedure to self-consistently solve for the kernel order parameters.