Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM

Neural networks, while effective for tackling many challengi ng scientific tasks, are not known to perform well out-of-distribution (OOD), i.e., within domains which d iffer from their training data. Understanding neural network OOD generalization is paramount to their suc cessful deployment in experimental workflows, especially when ground-truth knowledge about the experime nt is hard to establish or experimental conditions significantly vary. With inherent access to ground-truth in formation and fine-grained control of underlying distributions, simulation-based data curation facilitate s precise investigation of OOD generalization behavior. Here, we probe generalization with respect to imaging condi tions of neural network segmentation models for high-resolution transmission electron microscopy (HRTEM) imaging of nanoparticles, training and measuring the OOD generalization of over 12,000 neural networks using synthetic data generated via random structure sampling and multislice simulation. Using the HRTEM contra st transfer function, we further develop a framework to compare information content of HRTEM datasets an d quantify OOD domain shifts. We demonstrate that neural network segmentation models enjoy significant performance stability, but will smoothly and predictably worsen as imaging conditions shift from the training distribution. Lastly, we consider limitations of our approach in explaining other OOD shifts, s uch as of the atomic structures, and discuss complementary techniques for understanding generalizatio n in such settings.