Supplementary Material for HandMeThat: Human-Robot Communication in Physical and Social Environments Y anming Wan