With the success of deep learning, there are growing concerns over interpretability (Lipton, 2018). Ideally, the explanation should be both faithful (reflecting the model's actual behavior) and plausible

However,transfer learningis quickly becoming mainstream practice in deep learning.

R(h). (23) Here for simplicity, we abused the symbolD in(22)by maximizing outh0 in the originalD. In the top-left areaP,suppose only oneexample (markedbyxwith vertical coordinate1)isconfidently labeled as positive, and the rest examples are highly inconfidently labeled, hence not to contribute to the riskR. Similarly,there isonly one confidently labeled example ()inthe bottom-right area ofP, and it is negative with vertical coordinate 1. Wheneverλ > 2, the optimalhλ is in(0,1)and can be solved by a quadratic equation. In contrast,di-MDD is immune to this problem becauseRis used only to determineh, while the di-MDD value itself is solely contributed byD. Same as the scenario of largeλ, we do not change the feature distribution of source and target domains, hence keepingD(h) = 1 |h|.

However,therobustnessand better performance of the proposed approaches result in a method, applicable to all the settings without worrying about the embedding size.

However,theygenerally lose performance inmore realistic scenarios like learning in a continual manner. In contrast, humans can incorporate their prior knowledge to learn new concepts efficiently without forgetting older ones. In this work, we leverage meta-learning to encourage the model to learn how to learn continually. Inspired by human concept learning, we develop agenerative classifier that efficiently uses data-drivenexperience tolearn newconcepts even from fewsamples while being immune to forgetting. Along with cognitiveand theoretical insights, extensiveexperiments onstandard benchmarks demonstrate the effectiveness of the proposed method.

Besides, we also find that the same pre-trained model can have very different rankingsondifferenttasks. Thepeak memory costofthis phase is 61MB under resolution 224, which is reached when the largest sub-network is sampled.

This is achieved through a game where we learn features that are both predictive and robust (insensitive to adversarial attacks), i.e. cannot beused todiscriminate between natural andadversarial data.