How to Tell Deep Neural Networks What We Know