[D] Learning to forget. Optimizing a confusion loss to remove bias. • r/MachineLearning

A few months ago I stumbled onto an interesting idea while listening to the TWiML & AI podcast. It described a process by which one could attempt to introduce confusion into a network (starting at any arbitrary hidden layer) so that it couldn't learn from select biases in the training data. For example, if you were training an image classification network, and you wanted to forbid the network to learn anything about race, you could use this technique, to do so. The problem is that I can't for the life of me remember what this technique is called, or what episode of the podcast it was discussed in. All that I remember is that I believe that the method proposed involved choosing a layer beyond which you didn't want the network to be able to include information about the bias you were trying to remove (the layer becomes a filter of sorts), and using that layer as input to a second neural network that was optimizing a confusion loss.