Comparing K-Nearest Neighbors and Potential Energy Method in classification problem. A case study using KNN applet by E.M. Mirkes and real life benchmark data sets

Abstract: K-nearest neighbors (KNN) method is used in many supervised learning classification problems. Potential Energy (PE) method is also developed for classification problems based on its physical metaphor. The energy potential used in the experiments are Yukawa potential and Gaussian Potential. In this paper, I use both applet and MATLAB program with real life benchmark data to analyze the performances of KNN and PE method in classification problems. The results show that in general, KNN and PE methods have similar performance. In particular, PE with Yukawa potential has worse performance than KNN when the density of the data is higher in the distribution of the database. When the Gaussian potential is applied, the results from PE and KNN have similar behavior. The indicators used are correlation coefficients and information gain. Keywords: K-nearest neighbor, potential energy method, Yukawa potential, Gaussian potential, correlation coefficients, information gain 1. Introduction The target of supervised learning is to learn a mapping from the input to an output whose correct values are provided. However for unsupervised learning, no correct values are provided hence the only known object is the input data and the target is to find the regularities in the input. Classification is considered as an object of supervised learning.