Grounding Graph Network Simulators using Physical Sensor Observations