Reviews: Incremental Scene Synthesis

Reading the rebuttal and the promised improvements to the writing have increased my score to a 7. ---------------------- The paper presents a spatially-structured memory model capable of registering observations onto a globally consistent map, localizing incoming data and hallucinating areas of the map not yet or partially visited. Although borrowing architectural details from previous work especially with respect to MapNet, the paper proposes a way to incorporate a generative process directly into a spatially structured memory. Previous generative models for scenes have omitted any spatial inductive bias, and present the model directly with the sequence of observations. Additionally, previous spatial architectures often assume the setting where an oracle localizer is available. The proposed architecture provides the generative model with strong geometric priors, which enable it to perform localization without needing an oracle and accurate view generation.