Reviews: SPoC: Search-based Pseudocode to Code

Originality Improving the translation procedure by leveraging error detection on the target programming language is not a new idea, even though the authors do not seem to be aware of this. In semantic parsing, this has been used quite a bit over the last year, under the name of "execution-guided decoding" (see for example the WikiSQL leaderboard, where the top 4 entries currently use this). I believe this first appeared in "Execution-Guided Neural Program Decoding" (Wang et al, 2018); the idea is closest to the prefix-based pruning idea in this submission. This leaves the new "multiclass classification" error localization technique as original contribution, as well as the new dataset. Clarity The core contribution of the paper is a new algorithm, which is only described in prose; I would have preferred pseudo-code here, which would have been more concise.