feature engineering and over-fitting • /r/MachineLearning

Over-fitting refers to almost "remembering" the exact data points, rather than learning an intelligent representation of the data. With a neural network of 10.000 hidden units, I can definitely overfit a trainset of 10.000 samples. Simply every hidden neuron can correspond to one input sample. Feature engineering concerns the expansion of your input space. Say you have input vectors of 20 features.