The Devil and the Network: What Sparsity Implies to Robustness and Memory

Robustness is a commonly bruited property of neural networks; in particular, a folk theorem in neural computation asserts that neural networks-in contexts with large interconnectivity-continue to function efficiently, albeit with some degradation, in the presence of component damage or loss. A second folk theorem in such contexts asserts that dense interconnectivity between neural elements is a sine qua non for the efficient usage of resources. These premises are formally examined in this communication in a setting that invokes the notion of the "devil"