Generating Random Parameters in Feedforward Neural Networks with Random Hidden Nodes: Drawbacks of the Standard Method and How to Improve It

The standard method of generating random weights and biases in fe edfor-ward neural networks with random hidden nodes, selects them bot h from the uniform distribution over the same fixed interval. In this work, we sh ow the drawbacks of this approach and propose a new method of generat ing random parameters. This method ensures the most nonlinear fragments o f sigmoids, which are most useful in modeling target function nonlinearity, are k ept in the input hypercube. In addition, we show how to generate activation f unctions with uniformly distributed slope angles. Keywords: Feedforward neural networks, Neural networks with random hidden nodes, Randomized learning algorithms 1. Introduction Single-hidden-layer feedforward neural networks with random hid den nodes (FNNRHN) have become popular in recent years due to their fast lea rning speed, good generalization performance and ease of implementatio n.