Must-Know: What is the curse of dimensionality?

Editor's note: This post was originally included as an answer to a question posed in our 17 More Must-Know Data Science Interview Questions and Answers series earlier this year. The answer was thorough enough that it was deemed to deserve its own dedicated post. "As the number of features or dimensions grows, the amount of data we need to generalize accurately grows exponentially." Let's take an example below. Figure 1 (a) shows 10 data points in one dimension i.e. there is only one feature in the data set.