Quantifying the Importance of Data Alignment in Downstream Model Performance