Reviews: Residual Networks Behave Like Ensembles of Relatively Shallow Networks

I think the experiments in this paper are very interesting and reveal interesting properties of Resnet. However, I don't think the claim that ResNets are "exponential ensembles" has been made adequetely. If the claim were weakened to "behave like exponential ensembles", this would seem to be reasonably well supported by the experiments, and it's also an interesting result. But I think the question of why this happens is still open. One thing that stands out is that no definition of "ensemble" is ever given, not even a rough definition.