Supplementarymaterial

The visited node contains a panoramic image and the corresponding feature is derived through a mean pooling operation. The action node contains the rawfeature from the corresponding image andtheangleinformation. Theplanner concatenates thelanguage encodingc and agent statesht with the node features to acquire the contextual information. The final node information {vt} is planned through several steps of message passing operations.