In this paper we explore how machine learning techniques can be applied to the discovery of efficient mathematical identities. We introduce an attribute grammar framework for representing symbolic expressions. Given a grammar of math operators, we build trees that combine them in different ways, looking for compositions that are analytically equivalent to a target expression but of lower computational complexity. However, as the space of trees grows exponentially with the complexity of the target expression, brute force search is impractical for all but the simplest of expressions. Consequently, we introduce two novel learning approaches that are able to learn from simpler expressions to guide the tree search. The first of these is a simple n -gram model, the other being a recursive neural-network. We show how these approaches enable us to derive complex identities, beyond reach of brute-force search, or human derivation.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper present a heuristic for node selection in Branch and Bound for Mixed Integer Programs based on machine learning. Although machine learning used for node selection is not new the paper present a new approach (to the best of my knowledge). They utilize a classifier together with an oracle for training two aspects: a node selection policy and a node pruning policy. The first one is used to enforce a linear order/priority on the current open nodes of the Branch and Bound while the second one is used to further shrink the list of open nodes by pruning the unpromising ones.

Branch-and-bound is a widely used method in combinatorial optimization, including mixed integer programming, structured prediction and MAP inference. While most work has been focused on developing problem-specific techniques, little is known about how to systematically design the node searching strategy on a branch-and-bound tree. We address the key challenge of learning an adaptive node searching order for any class of problem solvable by branch-and-bound. Our strategies are learned by imitation learning. We apply our algorithm to linear programming based branch-and-bound for solving mixed integer programs (MIP). We compare our method with one of the fastest open-source solvers, SCIP; and a very efficient commercial solver, Gurobi. We demonstrate that our approach achieves better solutions faster on four MIP libraries.

To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap between GNN theory and the theory of distributed local algorithms.

Neural Information Processing Systems http://nips.cc/

Causal discovery from observational and interventional data is a fundamental problem and prevalent in multiple areas of science and engineering (Pearl, 2009; Spirtes et al., 2000; Peters et al., 2017). Learning the underlying causal mechanisms is essential for policy design.

We thank the four reviewers for their insightful comments and suggestions. I looked into the paper in ref[12] . . . ": In [12], the greedy algorithm is generic, with no assumptions about models ": Random search leads to a set of For Tab. 1, we ran the Wilcoxon signed-rank test (paired along settings, datasets and model types) and For Tab. 2 (with more costly experiments), we do not have enough runs to apply such We nonetheless report the standard errors in the paper, which seem to indicate significant improvements. ": Those numbers indicate the size of the ensemble; we will clarify this point. ": We thank R1 for the idea and ran our entire benchmark for ResNet-20: ": Hyper ensembles can indeed be viewed as a mixture They typically use Bayes nonparametric priors/posteriors and MCMC; we use mixtures and SGD. ": When used with replacement, the greedy algorithm from Caruana et al. [12, Sec.

Neural Information Processing Systems http://nips.cc/