Efficient Unified Caching for Accelerating Heterogeneous AI Workloads