Evolutionary Robustness Checking in the Artificial Anasazi Model

Using the well-known Artificial Anasazi simulation for a case study, we investigate the use of genetic algorithms (GAs) for performing two common tasks related to robustness checking of agent-based models: parameter calibration and sensitivity analysis. In the calibration task, we demonstrate that a GA approach is able to find parameters that are equally good or better at minimizing error versus historical data, compared to a previous factorial grid-based approach. The GA approach also allows us to explore a wider range of parameters and parameter settings. Previous univariate sensitivity analysis on the Artificial Anasazi model did not consider potentially complex/nonlinear interactions between parameters. With the GA-based approach, we perform multivariate sensitivity analysis to discover how greatly the model can diverge from historical data, while the parameters are constrained within a close range of previously calibrated values. We show that by varying multiple parameters within a 10% range, the model can produce dramatically and qualitatively different results, and further demonstrate the utility of sensitivity analysis for model testing, by the discovery of a small coding error. Through this case study, we discuss some of the issues that can arise with calibration and sensitivity analysis of agent-based models.