Reviews: Online Structure Learning for Feed-Forward and Recurrent Sum-Product Networks

This paper proposed an online learning algorithm for static and dynamic sum-product networks (SPNs), a type of probabilistic model with tractable inference. The authors essentially combine local structure search in SPNs with a hard variant of expectation-maximization [1]. The algorithm maintains empirical covariance estimates of product nodes and leverages statistical dependence tests to decide when to replace a product (factorized distribution) with either a new leaf or a mixture (sum node). The algorithm further includes a pruning mechanism in order to trim over-grown structures. The proposed method is called online Structure Learning with Running Average Update (oSLRAU).