Boosting In-Context Learning with Factual Knowledge