Relevant sparse codes with variational information bottleneck

Matthew Chalk, Olivier Marre, Gasper Tkacik

Neural Information Processing Systems 

In many applications, it is desirable to extract only the relevant aspects of data. A principled way to do this is the information bottleneck (IB) method, where one seeks a code that maximizes information about a'relevance' variable,