Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning