Some of the most successful knowledge graph embedding (KGE) models for link prediction - CP, RESCAL, TUCKER, COMPLEX - can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits - constrained computational graphs allowing efficient marginalisation. Then, we design two recipes to obtain efficient generative circuit models by either restricting their activations to be non-negative or squaring their outputs. Our interpretation comes with little or no loss of performance for link prediction, while the circuits framework unlocks exact learning by MLE, efficient sampling of new triples, and guarantee that logical constraints are satisfied by design.

Neural Information Processing Systems http://nips.cc/

's weights define an exponential family.

Papandreou and Yuille[4] proposed the Perturb-and-MAP Random Field, an efficient sampling method forgeneral MarkovRandom Field. We compare the detailed structure of gradients of each estimator. All gradients are formed as a summation over the steps. The Gumbel-CRF and PM-MRF estimator can be decomposed with a pathwise term, where we take gradientoff w.r.t. Since the official test set is not publically available, we use the same training/ validation/ test split as Fu et al.[1].