Modeling Dynamic Environments with Scene Graph Memory