Using big data and artificial intelligence to accelerate global development

When U.N. member states unanimously adopted the 2030 Agenda in 2015, the narrative around global development embraced a new paradigm of sustainability and inclusion--of planetary stewardship alongside economic progress, and inclusive distribution of income. This comprehensive agenda--merging social, economic and environmental dimensions of sustainability--is not supported by current modes of data collection and data analysis, so the report of the High-Level Panel on the post-2015 development agenda called for a "data revolution" to empower people through access to information.1 Today, a central development problem is that high-quality, timely, accessible data are absent in most poor countries, where development needs are greatest. In a world of unequal distributions of income and wealth across space, age and class, gender and ethnic pay gaps, and environmental risks, data that provide only national averages conceal more than they reveal. This paper argues that spatial disaggregation and timeliness could permit a process of evidence-based policy making that monitors outcomes and adjusts actions in a feedback loop that can accelerate development through learning. Big data and artificial intelligence are key elements in such a process. Emerging technologies could lead to the next quantum leap in (i) how data is collected; (ii) how data is analyzed; and (iii) how analysis is used for policymaking and the achievement of better results. Big data platforms expand the toolkit for acquiring real-time information at a granular level, while machine learning permits pattern recognition across multiple layers of input. Together, these advances could make data more accessible, scalable, and finely tuned. In turn, the availability of real-time information can shorten the feedback loop between results monitoring, learning, and policy formulation or investment, accelerating the speed and scale at which development actors can implement change.