A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

Abstract--Single-hidden layer feed forward neural networks (SLFNs) are widely used in pattern classification problems, but a huge bottleneck encountered is the slow speed and poor perf or-mance of the traditional iterative gradient-based learnin g algorithms. Although the famous extreme learning machine (ELM) has successfully addressed the problems of slow convergenc e, it still has computational robustness problems brought by inp ut weights and biases randomly assigned. Thus, in order to over - come the aforementioned problems, in this paper, a novel typ e neural network based on Gegenbauer orthogonal polynomials, termed as GNN, is constructed and investigated. This model c ould overcome the computational robustness problems of ELM, whi le still has comparable structural simplicity and approximat ion capability. Based on this, we propose a regularized weights direct determination (R-WDD) based on equality-constrain ed optimization to determine the optimal output weights. The R - WDD tends to minimize the empirical risks and structural ris ks of the network, thus to lower the risk of over fitting and impro ve the generalization ability. This leads us to a the final GNN wi th R-WDD, which is a unified learning mechanism for binary and multi-class classification problems. Finally, as is verifie d in the various comparison experiments, GNN with R-WDD tends to have comparable (or even better) generalization performan ces, computational scalability and efficiency, and classificati on robustness, compared to least square support vector machine ( LS-SVM), ELM with Gaussian kernel. ESEARCHES on artificial feed-forward neural networks (FNNs) have become increasingly active and popular, for it is one of the most powerful tools in artificial intelligenc e field.