Improving the Distributional Alignment of LLMs using Supervision