Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)

A plethora of aspects on the robustness have been studied, ranging from algorithms to their initialization as well as from the width of neural networks to their depth (i.e., the architecture).