Compute Or Load KV Cache? Why Not Both?