An Efficient Private GPT Never Autoregressively Decodes