We address the problem of fitting 3D human models to 3D scans of dressed humans. Classical methods optimize both the data-to-model correspondences and the human model parameters (pose and shape), but are reliable only when initialised close to the solution. Some methods initialize the optimization based on fully supervised correspondence predictors, which is not differentiable end-to-end, and can only process a single scan at a time. Our main contribution is LoopReg, an end-to-end learning framework to register a corpus of scans to a common 3D human model. The key idea is to create a self-supervised loop.

Weaknesses: This is not a weakness per-say, but a suggestion to make the paper stronger. In juxtaposition to the existing work the authors present the argument several times that using a UV parameterization is inherently inferior to 3D representations, as it requires seam-cuts and results in distortion of highly curved regions, etc. While this is conceptually correct and true, it would have made the paper stronger if the authors had somehow demonstrated this to be true empirically as well for their problem. For example, perhaps via a simpler problem -- maybe for the fully-supervised case or for the case when the entire pipeline is not necessarily end-to-end differentiable, but a combination of a landmarks/correspondence estimation a traditional optimization approach. It would be interesting to see if the signed distance representation to predict correspondences with a CNN along with its Lagrangian loss formulation to encourage points to lie on the surface improves the accuracy of correspond prediction by itself and if so by how much versus an approach that learns to map scan points to the UV space instead.

The rebuttal addressed the main criticisms raised by the reviewers: the assumption on warm start, the robustness to noise, and the clarification of the model. The answers of the authors contributed to the discussion and the proper evaluation of this work. The terminological issue doesn't affect the final decision.

We address the problem of fitting 3D human models to 3D scans of dressed humans. Classical methods optimize both the data-to-model correspondences and the human model parameters (pose and shape), but are reliable only when initialised close to the solution. Some methods initialize the optimization based on fully supervised correspondence predictors, which is not differentiable end-to-end, and can only process a single scan at a time. Our main contribution is LoopReg, an end-to-end learning framework to register a corpus of scans to a common 3D human model. The key idea is to create a self-supervised loop.