Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing