Big Data and Machine Learning Won't Save Us from Another Financial Crisis

Ten years on from the financial crisis, stock markets are regularly reaching new highs and volatility levels new lows. The financial industry has enthusiastically and profitably embraced big data and computational algorithms, emboldened by the many triumphs of machine learning. However, it is imperative we question the confidence placed in the new generation of quantitative models, innovations which could, as William Dudley warned, "lead to excess and put the [financial] system at risk." Eighty years ago, John Maynard Keynes introduced the concept of irreducible uncertainty, distinguishing between events one can reasonably calculate probabilities for, such as the spin of a roulette wheel, and those which remain inherently unknown, such as war in ten years' time. Today, we face the risk that investors, traders, and regulators are failing to understand the extent to which technological progress is -- or more precisely is not -- reducing financial uncertainty.