Reviews: Learning Deep Parsimonious Representations

This is a very solid paper all around. The idea to cluster activations and force the network to "stay close" to a restricted set of representations is intuitive, acting as a parsimony constraint that additionally enables interpretability. While solid theoretical motivation and analysis isn't provided, the proposed algorithm feels natural and the experiments are fairly comprehensive and compelling. A broad range of tasks are considered, including unsupervised, fine-grained, and zero-shot learning in addition to standard classification, and the visualizations of cluster structure demonstrate that the cluster centers are meaningful---they convince me that the approach is performing as one would hope. Some specific points: It would be nice to see a table or chart demonstrating the effect of different choices of cluster size on generalization error (it is mentioned that cross-validation was used, but it would still be good to get a feel for the sensitivity with respect to choice of cluster size).