All models were implemented in PyTorch. For all experiments, we report standard error below. Primitive rules map a word to a color (e.g. In a higher-order rule, the left hand side can be one or two variables and a word, and the right hand side can be any sequence of bracketed forms of those variables. Figure A.2 shows several example training grammars sampled from the meta-grammar.

We would like to thank all three reviewers for their thoughtful comments. R3 saw our approach as "very similar to the standard approach for neural We believe our model actually differs significantly from previous approaches in this regard. While our code is able to perform a "double attention" mechanism, this work does not use these features of We thank R1 and apologize for this confusion. According to R2, our paper "shows quite convincingly that neural program Our revision will report this experiment and move the discussion on the heuristics to the main text. Our approach utilizes test-time search, which R3 also suggests is a disadvantage: "The results of [the no search In that sense, our approach offers more robustness than a neural-only model would allow. The reviewers note that our model uses strong supervision in the form of a meta-grammar. In a sense, we agree with R2: "Now that this paper has shown This demonstrates both generalization and graceful degradation on grammars with 3x the number of rules vs training.

Perception and reasoning are two representative abilities of intelligence that are integrated seamlessly during human problem-solving processes.

Thank you for pointing out the issue, we'll add more discussion in the revision.

We study the task of smoothing a circuit, i.e., ensuring that all children of a

We thank the reviewers for the insightful and helpful comments. Minor remarks will be reflected in the text. The reviewer raises the very relevant issue of applicability, both theoretical and practical. Also, the practical applicability of our work depends on a reasoner for some relevant fragment of first order logic. The improvements made to reasoners, e.g.

In this paper we explore how machine learning techniques can be applied to the discovery of efficient mathematical identities. We introduce an attribute grammar framework for representing symbolic expressions. Given a grammar of math operators, we build trees that combine them in different ways, looking for compositions that are analytically equivalent to a target expression but of lower computational complexity. However, as the space of trees grows exponentially with the complexity of the target expression, brute force search is impractical for all but the simplest of expressions. Consequently, we introduce two novel learning approaches that are able to learn from simpler expressions to guide the tree search. The first of these is a simple n -gram model, the other being a recursive neural-network. We show how these approaches enable us to derive complex identities, beyond reach of brute-force search, or human derivation.

Neural Information Processing Systems http://nips.cc/

We introduce an unsupervised learning algorithm that combines probabilistic modeling with solver-based techniques for program synthesis. We apply our techniques to both a visual learning domain and a language learning problem, showing that our algorithm can learn many visual concepts from only a few examples and that it can recover some English inflectional morphology. Taken together, these results give both a new approach to unsupervised learning of symbolic compositional structures, and a technique for applying program synthesis tools to noisy data.

Primitives are expressions in typed lambda calculus.