Backward baselines: Is your model predicting the past?

Proponents of predictive technologies for consequential decision-making emphasize the seeming ability of statistical models to anticipate future outcomes. The ability to predict the future, so the argument goes, creates a rationale for adopting machine learning as policy: if a risk score charted the future trajectory of individuals, then intervening in a person's life on the basis of the score would be justified [KLMO15, OE16]. At the same time, critical scholars caution that predictive technologies reproduce historical patterns of injustice and social stratification. In this account, rather than predicting future outcomes, statistical risk assessment tools punish individuals for factors predating their own agency [Eub18, Ben19]. Does a statistical model predict the future or recite the past? The answer to the question is often not obvious. Consider the problem of loan default prediction, one of many tasks often framed as predicting future outcomes. A forward-looking predictor might identify individual behavior detrimental to loan repayment and adjust the predicted likelihood of default accordingly. Alternatively, a backward-looking predictor might take note of historical associations between repayment and demographic factors, then predict based solely on the historical factors.