Low Latency Privacy Preserving Inference