Bias and Variance -- Cut Through the Noise

Bias and Variance are amongst the more misunderstood concepts in ML, as they're usually described using superficial explanations of under-fitting and over-fitting. In this post, we lay down the statistical groundwork to understand where they come from. The maths is thoroughly explained, so you won't need to be an expert in Statistics to understand it. That said, you should be familiar with the basic concepts of a probability distribution and its Expected Value (average). We'll also assume you're familiar with the ML ideas of regression and supervised learning.