Patch Rebirth: Toward Fast and Transferable Model Inversion of Vision Transformers