Rethinking Timing Residuals: Advancing PET Detectors with Explicit TOF Corrections