Best Practices and Lessons Learned on Synthetic Data for Language Models