Supervised Knowledge Makes Large Language Models Better In-context Learners