Stochastic Generative Flow Networks

Contrary to the typical reward-maximizing policy Generative Flow Networks (or GFlowNets for in RL [Mnih et al., 2015, Lillicrap et al., 2015, Haarnoja short) are a family of probabilistic agents that et al., 2017, Fujimoto et al., 2018, Haarnoja et al., 2018], learn to sample complex combinatorial structures GFlowNets aim to learn a stochastic policy for sampling through the lens of "inference as control". They composite objects x with probability proportional to the have shown great potential in generating highquality reward function R(x). This is desirable in many real-world and diverse candidates from a given energy tasks where the diversity of solutions is important, and we landscape. However, existing GFlowNets can be aim to sample a diverse set of high-reward candidates, including applied only to deterministic environments, and recommender systems [Kunaver and Požrl, 2017], fail in more general tasks with stochastic dynamics, drug discovery [Bengio et al., 2021a, Jain et al., 2022a], and which can limit their applicability. To overcome sampling causal models from a Bayesian posterior [Deleu this challenge, this paper introduces Stochastic et al., 2022], among others. GFlowNets, a new algorithm that extends GFlowNets to stochastic environments. By decomposing state transitions into two steps, Stochastic GFlowNets isolate environmental stochasticity and learn a dynamics model to capture it. Extensive experimental results demonstrate that Stochastic GFlowNets offer significant advantages over standard GFlowNets as well as MCMC-and RL-based approaches, on a variety of standard benchmarks Figure 1: An example illustrating the failure of existing with stochastic dynamics.