Machine Learning: What Counting Jelly Beans Can Teach Us

Remember that old carnival game, the one where you attempt to guess the number of jelly beans in a jar? While it often took some combination of luck and skill for any single person to accurately guess the correct number, it turns out that by averaging all of the guesses of a wide variety of people together, the averaged answer is surprisingly close to the correct response. This phenomenon is an example of what's known as "the wisdom of the crowd," a modeling strategy frequently used in machine learning. Given that you have a diverse enough number of perspectives--each of which must have some measure of signal, but not be correlated to any other perspective (so errors tend to be symmetrically distributed around the truth)--as well as a suitable way of aggregating those perspectives (like averaging), you'll find that in the results of that aggregation, the "rightness stacks up" while the errors tend to cancel each other out. In the case of the jelly bean example, this means you must have a lot of people submit guesses (large number of perspectives), they're all looking at the same jar of jelly beans (must have some measure of signal), and those people can't talk to each other about their guesses (perspectives are not otherwise correlated).