Boosting In-Context Learning in LLMs Through the Lens of Classical Supervised Learning