Sample Size Requirements for Feedforward Neural Networks

We investigate the tradeofi"s among network complexity, training set size, and sta(cid:173) tistical performance of feedforward neural networks so as to allow a reasoned choice of network architecture in the face of limited training data. Nets are functions 7](x; w), parameterized by their weight vector w E W Rd, which take as input points x E Rk. For classifiers, network output is restricted to {a, 1} while for fore(cid:173) casting it may be any real number. The architecture of all nets under consideration is N, whose complexity may be gauged by its Vapnik-Chervonenkis (VC) dimension v, the size of the largest set of inputs the architecture can classify in any desired way ('shatter'). Nets 7] EN are chosen on the basis of a training set T {(Xi, YiHr l. These n samples are i.i.d.