Evaluation Challenges for Geospatial ML

As geospatial machine learning models and maps derived from their predictions are increasingly used for downstream analyses in science and policy, it is imperative to evaluate their accuracy and applicability. Geospatial machine learning has key distinctions from other learning paradigms, and as such, the correct way to measure performance of spatial machine learning outputs has been a topic of debate. In this paper, I delineate unique challenges of model evaluation for geospatial machine learning with global or remotely sensed datasets, culminating in concrete takeaways to improve evaluations of geospatial model performance. Geospatial machine learning (ML), for example with remotely sensed data, is being used across consequential domains, including public health (Nilsen et al., 2021; Draidi Areed et al., 2022) conservation (Sofaer et al., 2019), food security (Nakalembe, 2018), and wealth estimation (Jean et al., 2016; Chi et al., 2022). By both their use and their very nature, geospatial predictions have a purpose beyond model benchmarking; mapped data are to be read, scrutinized, and acted upon.