Zero-shot semantic segmentation aims to recognize the semantics of pixels from unseen categories with zero training samples. Previous practice [1] proposed to train the classifiers for unseen categories using the visual features generated from semantic word embeddings. However, the generator is merely learned on the seen categories while no constraint is applied to the unseen categories, leading to poor generalization ability. In this work, we propose a Consistent Structural Relation Learning (CSRL) approach to constrain the generating of unseen visual features by exploiting the structural relations between seen and unseen categories. We observe that different categories are usually with similar relations in either semantic word embedding space or visual feature space.

Summary and Contributions: Post rebuttal update I originally gave this paper an '8' and I will keep my original rating. The method is a good improvement upon [1]: it extends [1] with a simple and reproducable idea. Experimentally they demonstrate good improvements over [1]. In contrast to R3, I think that is not only a decent amount of novelty, but also the simple kind of novelty that is likely to be adopted by other reviewers. The other two main weaknesses highlighted by several reviewers were: 1) A better positioning w.r.t.

Paper originally received a set of somewhat mixed reviews from four reviewers, with scores: 8, 5, 5, 6. Generally, the reviewers liked the work, commenting on how it addressed an important problem [R3] and presented a well-motivated idea [R1] that was novel [R2], simple and reproducible [R1]; ultimately resulting in good results [R1,R2,R3,R4]. Some shortcoming were also identified, including (1) unclear positioning and potential limited novelty with respect to [1] [R1,R2,R3] and (2) lack of sufficient comparisons to related work [R2,R3,R4]. Authors have provided a very through rebuttal that addressed all major concerns; providing compelling clarification of novelty (1) and additional experiments to address reviews comments for (2). As a result R2 and R3 raised their scores arriving at the final unanimously positive ratings for the paper of: 8, 7, 6, 6. AC has read the reviews, the rebuttal, resulting discussion and the paper itself.

Zero-shot semantic segmentation aims to recognize the semantics of pixels from unseen categories with zero training samples. Previous practice [1] proposed to train the classifiers for unseen categories using the visual features generated from semantic word embeddings. However, the generator is merely learned on the seen categories while no constraint is applied to the unseen categories, leading to poor generalization ability. In this work, we propose a Consistent Structural Relation Learning (CSRL) approach to constrain the generating of unseen visual features by exploiting the structural relations between seen and unseen categories. We observe that different categories are usually with similar relations in either semantic word embedding space or visual feature space.