Promoting Poor Features to Supervisors: Some Inputs Work Better as Outputs