Goto

Collaborating Authors

 face reconstruction




Face Reconstruction from Facial Templates by Learning Latent Space of a Generator Network

Neural Information Processing Systems

In this paper, we focus on the template inversion attack against face recognition systems and propose a new method to reconstruct face images from facial templates. Within a generative adversarial network (GAN)-based framework, we learn a mapping from facial templates to the intermediate latent space of a pre-trained face generation network, from which we can generate high-resolution realistic reconstructed face images. We show that our proposed method can be applied in whitebox and blackbox attacks against face recognition systems. Furthermore, we evaluate the transferability of our attack when the adversary uses the reconstructed face image to impersonate the underlying subject in an attack against another face recognition system. Considering the adversary's knowledge and the target face recognition system, we define five different attacks and evaluate the vulnerability of state-of-the-art face recognition systems. Our experiments show that our proposed method achieves high success attack rates in whitebox and blackbox scenarios. Furthermore, the reconstructed face images are transferable and can be used to enter target face recognition systems with a different feature extractor model. We also explore important areas in the reconstructed face images that can fool the target face recognition system.


GLVD: Guided Learned Vertex Descent

Rico, Pol Caselles, Noguer, Francesc Moreno

arXiv.org Artificial Intelligence

Existing 3D face modeling methods usually depend on 3D Morphable Models, which inherently constrain the representation capacity to fixed shape priors. Optimization-based approaches offer high-quality reconstructions but tend to be computationally expensive. In this work, we introduce GLVD, a hybrid method for 3D face reconstruction from few-shot images that extends Learned Vertex Descent (LVD) by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. By incorporating relative spatial encoding, GLVD iteratively refines mesh vertices without requiring dense 3D supervision. This enables expressive and adaptable geometry reconstruction while maintaining computational efficiency. GLVD achieves state-of-the-art performance in single-view settings and remains highly competitive in multi-view scenarios, all while substantially reducing inference time.


Geometry-aware Two-scale PIFu Representation for Human Reconstruction

Neural Information Processing Systems

In a sparse ( e.g. , 3 RGBD sensors) capture Three-dimensional human reconstruction, which aims to obtain a dense surface geometry from single-view or multi-view human images, is a fundamental topic in computer vision and computer graphics. However, these methods typically only obtain minimally clothed human bodies.


Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space

Razzhigaev, Anton, Mikhalchuk, Matvey, Kireev, Klim, Udovichenko, Igor, Kuznetsov, Andrey, Petiushko, Aleksandr

arXiv.org Artificial Intelligence

Reconstructing facial images from black-box recognition models poses a significant privacy threat. While many methods require access to embeddings, we address the more challenging scenario of model inversion using only similarity scores. This paper introduces DarkerBB, a novel approach that reconstructs color faces by performing zero-order optimization within a PCA-derived eigenface space. Despite this highly limited information, experiments on LFW, AgeDB-30, and CFP-FP benchmarks demonstrate that DarkerBB achieves state-of-the-art verification accuracies in the similarity-only setting, with competitive query efficiency.


Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction

Giebenhain, Simon, Kirschstein, Tobias, Rünz, Martin, Agapito, Lourdes, Nießner, Matthias

arXiv.org Artificial Intelligence

We address the 3D reconstruction of human faces from a single RGB image. To this end, we propose Pixel3DMM, a set of highly-generalized vision transformers which predict per-pixel geometric cues in order to constrain the optimization of a 3D morphable face model (3DMM). We exploit the latent features of the DINO foundation model, and introduce a tailored surface normal and uv-coordinate prediction head. We train our model by registering three high-quality 3D face datasets against the FLAME mesh topology, which results in a total of over 1,000 identities and 976K images. For 3D face reconstruction, we propose a FLAME fitting opitmization that solves for the 3DMM parameters from the uv-coordinate and normal estimates. To evaluate our method, we introduce a new benchmark for single-image face reconstruction, which features high diversity facial expressions, viewing angles, and ethnicities. Crucially, our benchmark is the first to evaluate both posed and neutral facial geometry. Ultimately, our method outperforms the most competitive baselines by over 15% in terms of geometric accuracy for posed facial expressions.


Exploiting Multiple Representations: 3D Face Biometrics Fusion with Application to Surveillance

La Cava, Simone Maurizio, Casula, Roberto, Concas, Sara, Orrù, Giulia, Tolosana, Ruben, Drahansky, Martin, Fierrez, Julian, Marcialis, Gian Luca

arXiv.org Artificial Intelligence

3D face reconstruction (3DFR) algorithms are based on specific assumptions tailored to the limits and characteristics of the different application scenarios. In this study, we investigate how multiple state-of-the-art 3DFR algorithms can be used to generate a better representation of subjects, with the final goal of improving the performance of face recognition systems in challenging uncontrolled scenarios. We also explore how different parametric and non-parametric score-level fusion methods can exploit the unique strengths of multiple 3DFR algorithms to enhance biometric recognition robustness. With this goal, we propose a comprehensive analysis of several face recognition systems across diverse conditions, such as varying distances and camera setups, intra-dataset and cross-dataset, to assess the robustness of the proposed ensemble method. The results demonstrate that the distinct information provided by different 3DFR algorithms can alleviate the problem of generalizing over multiple application scenarios. In addition, the present study highlights the potential of advanced fusion strategies to enhance the reliability of 3DFR-based face recognition systems, providing the research community with key insights to exploit them in real-world applications effectively. Although the experiments are carried out in a specific face verification setup, our proposed fusion-based 3DFR methods may be applied to other tasks around face biometrics that are not strictly related to identity recognition.


Reviews: Face Reconstruction from Voice using Generative Adversarial Networks

Neural Information Processing Systems

This paper proposes a convolutional neural network based model to reconstruct a face from spoken speech. The training is done by using supervised GAN. The problem is novel, but the model itself is not so much as using encoder (or embedder) and decoder (or generator) is quite standard, and supervised GAN training has also been popularly used, so in that perspective, its novelty is incremental. But I think this paper needs more thorough experimental study to show the effectiveness of the proposed model: 1. From the experimental results, I suspect that the generated faces only match those attributes (gender, race, etc.) but not much about identities.


Reviews: Face Reconstruction from Voice using Generative Adversarial Networks

Neural Information Processing Systems

The paper proposes a very novel method that creates an estimate of a face from a voice and works as a supervised method . The reviewers initially were not so convinced and with some disagree. The rebuttal was satisfying so that also one reviewer changed its score from weak rejection to acceptance. Thus, after a discussion with the Senior Area chair, the paper is accepted . This meta-review was reviewed and revised by the Program Chairs, based on discussions with the Senior Area Chair.