Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection

--Remarkable advancements in generative AI technology have given rise to a spectrum of novel deepfake categories with unprecedented leaps in their realism, and deepfakes are increasingly becoming a nuisance to law enforcement authorities and the general public. In particular, we observe alarming levels of confusion, deception, and loss of faith regarding multimedia content within society caused by face deepfakes, and existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation. This is primarily due to their reliance on specific forgery artifacts, which limits their ability to generalise and detect novel deepfake types. T o combat the spread of malicious face deepfakes, this paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions while ensuring feature distinctiveness and reducing the redundancy of the modelled features. A novel feature orthogonality-based disentanglement strategy is introduced to ensure branch-level and cross-branch feature disentanglement, which allows us to integrate multiple feature vectors without adding complexity to the feature space or compromising generalisation. Comprehensive experiments on three public benchmarks: FaceForensics++, Celeb-DF, and the Deepfake Detection Challenge (DFDC) show that these design choices enable the proposed approach to outperform current state-of-the-art methods by 5% on the Celeb-DF dataset and 7% on the DFDC dataset in a cross-dataset evaluation setting. I NTRODUCTION The fake video published by BuzzFeed showing an apparent speech by former US President Barack Obama that was in fact performed by Jordan Peele [1] shows how easy it is to create convincing audio and video fakes. In recent years, we have seen an explosion of deep fakes, especially multimodal (video and audio) deep fakes. The extent and severe impact of fake multimedia content were clearly evident during the recent COVID-19 global pandemic [2] and the lead-up to the US federal 2020 election. Thus, the early detection of deep fakes is vital for stopping the spread of misinformation, which has influenced elections and led to serious consequences, including blackmail and fraud. To combat the surge of misleading deepfakes, a multitude of detection methods have emerged. However, there are significant concerns about whether these techniques can keep pace with the rapid advancements in deepfake generation [3], [4].