SupplementaryMaterialfor HandMeThat: Human-RobotCommunication inPhysicalandSocialEnvironments

In Section B, we summarize the statistics of the dataset. A.1 ObjectSpace Recall that HandMeThat uses an object-centric representation for states. Object hierarchy.HandMeThat classifies all categories into 5classes: location, receptacle, food, tool,andthing. Each class (except for"location") iscomposed ofmultiple subclasses, and each subclass contains several object categories. Intotal, there are155 object categories.