Grandpa Hates Robots - Interaction Constraints for Planning in Inhabited Environments