General Discussion Our work is part of the following larger and important discussion within the NeurIPS community: 2

We got a clear sense of where more clarification would be helpful. To what solution do neural nets (trained w. GD on this network simulates the unnormalized exponentiated gradient algorithm (EGU). Previously it was thought that GD cannot take advantage of the sparsity of the solution. What is the surprising insight?