Function Approximation with the Sweeping Hinge Algorithm

We present a computationally efficient algorithm for function ap(cid:173) proximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the method of fitting the residual. The task of fitting individual nodes is accom(cid:173) plished using a new algorithm that searchs for the best fit by solving a sequence of Quadratic Programming problems. Unique characteristics of this algorithm include: finite step convergence, a simple stop(cid:173) ping criterion, a deterministic methodology for seeking "good" local minima, good scaling properties and a robust numerical implemen(cid:173) tation.