This work has been extended to different geometries [7, 23] and various applications[60,9,13,24].

WeempiricallytestAPO for online adaptation of learning rates and structured preconditioning matrices for regression, image reconstruction, image classification, and natural language translationtasks.

We identify that the total error -- the sum of the compression errors for all iterations -- encapsulates sparsification throughout training.

A finite-state machine (FSM) is a computation model to process binary strings in sequential circuits.

Not surprisingly, we show that none of the assessed models are as robust as originally advertised in their respective papers.