Rediscovering Deep Neural Networks in Finite-State Distributions