On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists