A shooting formulation of deep learning

Indeed, existing works throw into relief the myriad difficulties of learning an infinite-dimensional parameter in a continuous-depth neural network. To this end, we introduce a shooting formulation which shifts the perspective from pa-rameterizing a network layer-by-layer to parameterizing over optimal networks described only by a set of initial conditions . For scalability, we propose a novel particle-ensemble parameterization which fully specifies the optimal weight trajectory of the continuous-depth neural network.