Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction