The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal

We analyze the Kozachenko-Leonenko (KL) fixed k -nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance for any fixed k over H older balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a recent mini-max lower bound over the H older ball, we show that the KL estimator for any fixed k is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter s of the H older ball for s (0, 2] and arbitrary dimension d, rendering it the first estimator that provably satisfies this property.