Private large language model (LLM) inference based on cryptographic primitives offers a promising path towards privacy-preserving deep learning. However, existing frameworks only support dense LLMs like LLaMA-1 and struggle to scale to mixture-of-experts (MoE) architectures. The key challenge comes from securely evaluating the dynamic routing mechanism in MoE layers, which may reveal sensitive input information if not fully protected. In this paper, we propose CryptoMoE, the first framework that enables private, efficient, and accurate inference for MoE-based models. CryptoMoE balances expert loads to protect expert routing information and proposes novel protocols for secure expert dispatch and combine. CryptoMoE also develops a confidence-aware token selection strategy and a batch matrix multiplication protocol to improve accuracy and efficiency further.

This work takes one step towards data encryption by incorporating some crucial ingredients of learning algorithm.

Neural Information Processing Systems http://nips.cc/

HE operations (e.g., ciphertext (ct) rotations/multiplications, additions), which could be orders of For example, a GCN layer's computation is dominated by the special consecutive HE operations are defined in Sec. 2. For generality, we assume both feature matrix and adjacency Parallel-Packing (see Sec. 3.2), the ciphertext size is fully exploited, and the total HE operation count We adopt a threat model setting consistent with prior works [9, 14, 3, 7, 18, 22, 27]. The cloud server is semi-honest (e.g.