We illustrate the architecture in Figure 1 in the main paper. We use a kernel size of 5. This is essentially an equivariant version of LayerNorm. In the geometric layers, the input state is split into scalar and vector components. The vector components are linearly transformed to reduce the number of channels to 16.

We provide additional architectural details here beyond those provided in the paper. In all models, the output layer consists of the computation of logits, followed by a softmax cross-entropy categorical loss term. Figure 6 provides the grammar. Figure 6: Grammar describing the generated programs comprising the dataset in this paper. Figure 8: The same programs as in Figure 7, with a single statement masked in each.

We thank the reviewers for their insightful comments. We next address questions and comments raised in the reviews. R1, R3: There is no ability to disentangle lighting and material, the paper is misleading in that aspect. In section 3.2 we will clearly state that, in theory, incorporating the surface This is in contrast to MVS pipelines (e.g., In some cases, such as the "Fountain" scene, our method can go beyond R1, R2: Training and inference times are missing. All relevant details will be added to the text.

We then optimize the prototypes with a fixed encoder/decoder.