Gaussian mixture models as a proxy for interacting language models