r/MachineLearning - [D] Visualizing and analyzing error landscapes

It's difficult to visualize and understand the high dimensional error landscapes (ie cost functions) of neural nets and other machine learning algorithms. A common method is to project the parameter space onto two dimensions and plot a surface. What are some effective choices for this projection that help visualize salient features of the error? Are there nonlinear approaches that are better? More importantly, what is known about the geometry of these cost functions for neural networks trained on real data?