What are Neural Networks made of?