local minimum
Local to Global: Learning Dynamics and Effect of Initialization for Transformers
In recent years, transformer-based models have revolutionized deep learning, particularly in sequence modeling.
Technology:
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.93)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.87)
Technology:
How Much Restricted Isometry is Needed In Nonconvex Matrix Recovery?
Richard Zhang, Cedric Josz, Somayeh Sojoudi, Javad Lavaei
Neural Information Processing Systems http://nips.cc/
Country:
- North America > United States > California > Alameda County > Berkeley (0.05)
- Asia > Middle East > Jordan (0.04)
- North America > Canada > Quebec > Montreal (0.04)
- Asia > China (0.04)
Technology:
A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization
Cedric Josz, Yi Ouyang, Richard Zhang, Javad Lavaei, Somayeh Sojoudi
We study the set of continuous functions that admit no spurious local optima (i.e.
Technology:
Adding One Neuron Can Eliminate All Bad Local Minima
SHIYU LIANG, Ruoyu Sun, Jason D. Lee, R. Srikant
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima.
Country:
- North America > United States > Illinois (0.05)
- North America > Canada > Quebec > Montreal (0.04)
Technology: Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.46)
Fixing Implicit Derivatives: Trust-Region Based Learning of Continuous Energy Functions
Chris Russell, Matteo Toso, Neill Campbell
Neural Information Processing Systems http://nips.cc/
Technology:
Country:
- Asia > China > Beijing > Beijing (0.04)
- Asia > British Indian Ocean Territory > Diego Garcia (0.04)
- Asia > China > Hong Kong (0.04)
- Asia > China > Guangdong Province > Guangzhou (0.04)
Technology:
Country:
- North America > United States > Illinois > Cook County > Evanston (0.04)
- North America > United States > New York > New York County > New York City (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Asia > Afghanistan > Parwan Province > Charikar (0.04)
Technology:
