A Realizable Learning Task which Exhibits Overfitting

In this paper we examine a perceptron learning task. The task is realizable since it is provided by another perceptron with identi(cid:173) cal architecture. Both perceptrons have nonlinear sigmoid output functions. The gain of the output function determines the level of nonlinearity of the learning task. It is observed that a high level of nonlinearity leads to overfitting.