BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling Lin Gui, Cristina Gârbacea 2, and Victor Veitch Department of Statistics, University of Chicago