Existing research on video representation for processing has primarily focused on 2D/2.5D

We perform computational experiments that support our theoretical results.

Neural Information Processing Systems http://nips.cc/

These authors contributed equally to this work.

Figure 2: Anexampleofadynamic (left) andstatic (right) neuralgraph. Claim: Assume L is Lipschitzcontinuous.

R2: Sparsity: Thank you for mentioning thatthe unification ofcoreparts ofthe sparseneuralnetwork literature14 combined with the neural architecture search problem is very nice. As a useful experiment to include to showcase15 this, we apply Algorithm 1 separately to each filter of the version of ResNet50 used in [1] and obtain state of the16 art performance atthe task oftraining sparse networks from scratchas described in [1]. R2: Theoretical Claims: We agree that this work will benefit from theoretical analysis which extends beyond25 decreasing the loss on the mini-batch with the SGD update. We thank you for the excellent suggestion of offering the additional motivation of31 performing an approximation in the backwards pass. We are happy to see that we may arrive at the same objec-32 tive in this way and we will include this exciting result.

This is meaningful because in most applications, the time complexities for evaluating gradients at different points are of the same magnitude.

Neural Information Processing Systems http://nips.cc/