Governments are increasingly turning to algorithmic risk assessments when making important decisions, believing that these algorithms will improve public servants' ability to make policy-relevant predictions and thereby lead to more informed decisions. Yet because many policy decisions require balancing risk-minimization with competing social goals, evaluating the impacts of risk assessments requires considering how public servants are influenced by risk assessments when making policy decisions rather than just how accurately these algorithms make predictions. Through an online experiment with 2,140 lay participants simulating two high-stakes government contexts, we provide the first large-scale evidence that risk assessments can systematically alter decision-making processes by increasing the salience of risk as a factor in decisions and that these shifts could exacerbate racial disparities. These results demonstrate that improving human prediction accuracy with algorithms does not necessarily improve human decisions and highlight the need to experimentally test how government algorithms are used by human decision-makers.
Artificial Intelligence and its inherent bias may not be as judgmental as previously thought, at least in the case of home loans. It appears the use of algorithms for online mortgage lending can reduce discrimination against certain groups, including minorities, according to a recent study from the National Bureau of Economic Research. This could end up becoming the main tool in closing the racial wealth gap, especially as banks start using AI for lending decisions. The Breakdown You Need to Know: The study found that in person mortgage lenders typically reject minority applicants at a rate 6% higher than those with comparable economic backgrounds. However, when the application was online and involved an algorithm to make the decision, the acceptance and rejection rates were the same.