Augmenting Interpretable Models with LLMs during Training