Understanding the Universal Approximation Theorem

Neural networks are one of the most beautiful programming paradigms ever invented. In the conventional approach to programming, we tell the computer what to do, breaking big problems up into many small, precisely defined tasks that the computer can easily perform. By contrast, in a neural network, we don't tell the computer how to solve our problem. Instead, it learns from observational data, figuring out its solution to the problem at hand. Until recently we didn't know how to train neural networks to surpass more traditional approaches, except for a few specialized problems.