Reviews: NAT: Neural Architecture Transformer for Accurate and Compact Architectures

However, the paper misses closely related work such as, e.g. However, it does not disentangle the effects of search space and search method; in particular, it remains unclear if the proposed and relatively complex search method (policy-gradient graph-convolutional neural networks) would outperform simpler baselines such as random search on the same search space. Moreover, the method is only applied to neural architectures that were not optimized for being resource-efficient; it remains unclear if NAT would also improve architectures such as, e.g., the MobileNet family or MnasNet. It also contains sufficient details for being able to replicate the paper. The proposed method could be seen as a post-processing step for NAS, which allows increasing the methods performance with the same or even less resources required (since the search space is very simple, a search method is more likely to find optimal configuration in it compared to the significantly larger typical NAS search spaces). However, to really show the significance of the proposed method, it would have to show that it outperforms simpler baselines such as random search and is also applicable to cells that were already optimized for being resource-efficient.