How can we use tools from signal processing to understand better neural networks?

Deep neural networks achieve state-of-the-art performance in many domains in signal processing. The main practice is getting pairs of examples, input, and its desired output, and then training a network to produce the same outputs with the goal that it will learn how to generalize also to new unseen data, which is indeed the case in many scenarios.