Synergy of Clustering Multiple Back Propagation Networks

The properties of a cluster of multiple back-propagation (BP) networks are examined and compared to the performance of a single BP network. Theunderlying idea is that a synergistic effect within the cluster improves the perfonnance and fault tolerance. Five networks were initially trainedto perfonn the same input-output mapping. Following training, a cluster was created by computing an average of the outputs generated by the individual networks. The output of the cluster can be used as the desired output during training by feeding it back to the individual networks.In comparison to a single BP network, a cluster of multiple BP's generalization and significant fault tolerance. It appear that cluster advantage follows from simple maxim "you can fool some of the single BP's in a cluster all of the time but you cannot fool all of them all of the time" {Lincoln} 1 INTRODUCTION Shortcomings of back-propagation (BP) in supervised learning has been well documented inthe past {Soulie, 1987; Bernasconi, 1987}. Often, a network of a finite size does not learn a particular mapping completely or it generalizes poorly.