Exact marginal prior distributions of finite Bayesian neural networks

Moreover, we show that the perturbative finite-width corrections do not capture the heavy-tailed nature of the true prior.