We thank the reviewers for providing detailed and quality reviews. A final novelty in our analysis is obtaining tighter bounds on the problem-dependent terms. We appreciate you pointing out several typos that we are fixing. Thanks for the detailed comments on the proof of Theorem 1. X and the index j is with respect to the m elements in Z . Thank you for the careful review and giving several useful suggestions.

We thank the reviewer for pointing out the typos in our paper.

T o Reviewer #3: Thank you for your careful reading and thoughtful reviews. Q1: Theorems 3 and 4. (i) Theorem 3: Theorem 3 shows that SD2 helps to reduce the overestimation bias compared We empirically show that SD2 does not underestimate and can reduce the absolute bias in Figure 4. The left-hand side in Eq. (19) equals to Q5: How is the performance of the proposed approximation method? We will try to further investigate it in future research. Q2: Related works about ensemble methods.

Post Rebuttal: Thank you for your response. I do see the advantages you listed to support the choice of language over programs. Overall, I feel the general direction of using language for intermediate supervision is really interesting and worthy of further study. This paper could be significantly improved however in some regards. For example: - Authors should study the generated language to test it for compositionality (as other reviewers have pointed out).