Export Reviews, Discussions, Author Feedback and Meta-Reviews

The manuscript proposes a formalism for computing stochastic estimates of gradients for loss functions. The formalism, referred to as stochastic computation graphs, is very general, applying to models with deterministic and stochastic components, and allowing the computation of gradient estimates for a broad range of models. Methods for deep networks, variational inference, and reinforcement learning are identified as special cases of the proposed framework. The proposed stochastic computation graphs are essentially Bayesian networks which may also contain deterministic nodes, and which are interpreted as encoding distributions over loss functions that are to be minimized. The gradient estimation algorithm extends backpropagation to the partially stochastic case covered by these models, by simply composing score function estimators and pathwise derivative estimators.