Fast Visuomotor Policies via Partial Denoising