Understanding Bias and Variance at abstract level

Bias and Variance are arguably the most important concepts in Machine Learning (ML). There is a lot of good ML literature that explains bias, variance and bias-variance trade-off. Also, often machine learning practitioners seem to believe that an increase in bias will surely increase variance and vice-versa. While this is probable, it is not always the case. This article is intended to explain bias and variance at an abstract level to ML enthusiasts with a belief that this knowledge will help them appreciate existing ML optimization techniques better. The job of any predictive machine learning algorithm is to estimate a function, as closely as possible, by looking at input and output of that function i.e. data.