We propose a reduction for non-convex optimization that can (1) turn an stationary-point finding algorithm into an local-minimum finding one, and (2) replace the Hessian-vector product computations with only gradient computations. It works both in the stochastic and the deterministic settings, without hurting the algorithm's performance. As applications, our reduction turns Natasha2 into a first-order method without hurting its theoretical performance. It also converts SGD, GD, SCSG, and SVRG into algorithms finding approximate local minima, outperforming some best known results.

The loss functions of deep neural networks are complex and their geometric properties are not well understood.

We show that most local optima of PNN optimizations are global while we have a characterization of regions where bad local optimizers may exist.

More complex data consisting of measurements across multiple modalities can be organized as higher dimensional data arrays, or higher order tensors.

Neural Information Processing Systems http://nips.cc/

Neural network training relies on our ability to find "good" minimizers of highly

Large amounts of labeled data are typically required to train deep learning models.

We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse.

Coordinated counterterrorism raids across Germany and Austria, with Mossad and European agencies working together, uncover Hamas network planning attacks on Israeli and Jewish civilians.

Neural Information Processing Systems http://nips.cc/