Frugal, Flexible, Faithful: Causal Data Simulation via Frengression

The use of machine learning tools has given causal inference a new lease of life, enabling complex models to be used with principled causal estimators and guarantees about statistically important quantities (Wager and Athey, 2018; Chernozhukov et al., 2018; Hahn et al., 2020). To build trustworthy causal models, however, we also need to understand when these methods may be more or less reliable, or perhaps fail completely. This implies that causal inference needs a set of good benchmarking tools. Unfortunately, real-world datasets are not ideal for this task, because they cannot give us access to the ground truth other than in a few very special circumstances. In particular, they rarely provide the counterfactual outcomes we care about, and the distribution we want to evaluate often differs from the one that produced the observations. Well-designed simulations can address this discrepancy (Neal et al., 2020; Parikh et al., 2022); they allow us to choose a ground truth, stress-test new methods, compare their generalizability and stability, and expose failure modes before deployment.