Tug the Student to Learn Right: Progressive Gradient Correcting by Meta-learner on Corrupted Labels

While deep networks have strong fitting capability to complex input patterns, they can easily overfit tobiased training data with corrupted labels. Sample reweighting strategy is commonly used to alleviate this robust learning issue, through imposing zeroor possibly smaller weights to corrupted samples to suppress their negative influence to learning. Current reweighting algorithms, however, needelaborate tuning of additional hyperparameters orcareful designing of complex metalearner for learning to assign weights on samples. To address these issues, we propose a new metalearning methodwith few tuned hyper-parameters and simple structure of a meta-learner (one hidden layer MLP network). Guided by a small amount of unbiased metadata, the parameters of the proposed meta-learnercan be gradually evolved for finely tugging the classifier gradient approaching tothe right direction. This learning manner complies with a real teaching progress: A good teacher should more respect the student's own learning manner and help progressively correct his learning bias based on his/her current learning status.Experimental results substantiate the robustness of the new algorithm on corrupted label cases,as well as its stability and efficiency in learning. The dream begins with a teacher who believes in you, who tugs and pushes and leads you to the next plateau, sometimes pokingyou with a sharp stick called truth.