Think Big, Teach Small: Do Language Models Distil Occam's Razor?