Fill In The Gaps: Model Calibration and Generalization with Synthetic Data