Routing Distilled Knowledge via Mixture of LoRA Experts for Large Language Model based Bundle Generation

--Large Language Models (LLMs) have shown potential in automatic bundle generation but suffer from prohibitive computational costs. Although knowledge distillation offers a pathway to more efficient student models, our preliminary study reveals that naively integrating diverse types of distilled knowledge from teacher LLMs into student LLMs leads to knowledge conflict, negatively impacting the performance of bundle generation. T o address this, we propose RouteDK, a framework for routing distilled knowledge through a mixture of LoRA expert architecture. We then train knowledge-specific LoRA experts for each type of knowledge together with a base LoRA expert. For effective integration, we propose a dynamic fusion module, featuring an input-aware router, where the router balances expert contributions by dynamically determining optimal weights based on input, thereby effectively mitigating knowledge conflicts. T o further improve inference reliability, we design an inference-time enhancement module to reduce variance and mitigate suboptimal reasoning. Experiments on three public datasets show that our RouteDK achieves accuracy comparable to or even better than the teacher LLM, while maintaining strong computational efficiency. In addition, it outperforms state-of-the-art approaches for bundle generation. RODUCT bundling is a critical merchandising strategy that groups a number of complementary or alternative items into a single package, applied in various domains such as e-commerce, retail, and telecommunications [1]-[3]. With this strategy, vendors can satisfy diverse customer needs, enhance user experiences, and drive increased sales and engagement, while users benefit from convenience and discounted price.