HETEROGENEOUSSWARMS: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems

We propose HETEROGENEOUSSWARMS, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. We represent multi-LLM systems as directed acyclic graphs (DAGs) of LLMs with topological message passing for collaborative generation.