Purifying Large Language Models by Ensembling a Small Language Model