Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

In many machine learning applications one optimizes a non-convex loss function; this is often achieved using simple descending algorithms such as gradient descent or its stochastic variations.