Infusing Commonsense World Models with Graph Knowledge