Reviews: Gradient Information for Representation and Modeling

This paper develops new information quantities. These quantities are derived from the Hyvarinen loss and are shown to be related to Fisher divergence. The behavior of some of these quantities is similar in part to classical information quantities such as entropy, mutual information, KL divergence etc. The former quantities are said to be faster to compute and more robust than the latter. In this paper, these quantities are derived, studied and used to derive several learning algorithms, most notably a version of the Chow-Liu algorithm based on these quantities.