Out of Bag (OOB) Evaluation in Random Forests

Out of Bag (OOB) Evaluation is a very important yet underrated topic in ensemble learning. People tend to learn a lot about Random forests and other bagging algorithms, but often they tend to skip or overlook this concept. I myself missed it while learning about ensemble models and failed an interview where the last question asked was "How are the Out of Bag data utilized while training a random forest model?" (hence, decided to write this blog as a lesson) Oops! Cannot recall random forests? Basically, it is nothing but absolute supervised learning based on the concept of creating independent base learners (multiple decision trees containing bootstrapped samples from the original dataset) and training them. The bootstrapped samples are created by random sampling with replacement of dataset(d), with n features, where each sample d is less than d, and n n.