Delving into Sequential Patches for Deepfake Detection