[D] KNN Performance decrease when new features were introduced
It's still quite useful, but it does not do any kind of feature selection nor does it consider any kind of feature importance when making predictions. All it does to make predictions is to calculate the distance in feature space between the new observation you wish to make a prediction for and all the other observations it has been trained on, and find the k closest old ones to the new one, then take some aggregate of the target variable of the k closest observations (usually mean for regression and mode for classification). If you add several completely random columns to your data, kNN will use them in calculating the distance to the exact same extent as the meaningful columns. This is opposed to smarter algorithms like linear models that can figure out to ignore features that contain no predictive value. If your model is getting worse when you add new features, it doesn't even mean they contain no value.
Mar-6-2021, 17:01:34 GMT
- Technology: