Palette of Language Models: A Solver for Controlled Text Generation