To Ask or Not to Ask: Learning to Require Human Feedback