Neural Network Bias: Bias Neuron, Overfitting and Underfitting -

Practically, when training a neural network model, you will attempt to gather a training set that is as large as possible and resembles the real population as much as possible. You will then break up the training set into at least two groups: one group will be used as the training set and the other as the validation set. Bias--high bias means the model is not "fitting" well on the training set. This means the training error will be large. Low bias means the model is fitting well, and training error will be low.