Appendix ABroaderSocietalImpact

Our intention is for this algorithm to be used in a real-world setting where humans can provide natural language instructions to robots that can carry them out. For the LOReL baseline, we used the numbers from the paper for LOReL Images. Unless otherwise specified, we use the following settings. D.4 MICalculation We calculate mutual information between the language instructions (L) and the skill codes (z) by writing MI(L,z)=H(L) H(L|z). RephrasalType Flat LISA seen 15 40 unseennoun 13.33 33.33 unseenverb 28.33 30 unseennoun+verb 6.7 20 human 26.98 27.35 Thismeansweare using a manual (human) skill predictor as opposed to our trained skill predictor.