A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs