Optimizing Privacy-Preserving Primitives to Support LLM-Scale Applications