Continuous vs. Discrete Optimization of Deep Neural Networks