A Survey on Universal Approximation Theorems

A neural network (NN) or artificial neural network (ANN) is a network of artificial neurons arranged in layers [1, 2]. The artificial neurons (also called perceptrons) are inspired by biological neurons in biological neural networks (BNNs)[3]. Biological neurons are the signal-processing units of BNN in the brain, similarly, artificial neurons are data-processing units in ANN. The rest of the paper discusses only ANN and artificial neurons which will be referred to simply by NN and neuron. From a mathematical point of view, neurons are made of compositions of a nonlinear function (also called activation function) and a linear function [4].