Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models Zhanhui Zhou, Yu Qiao Shanghai Artificial Intelligence Laboratory