Unraveling the Gradient Descent Dynamics of Transformers

Feb-17-2026, 06:30:27 GMT–Neural Information Processing Systems

By analyzing the loss landscape of a single Transformer layer using Softmax and Gaussian attention kernels, our work provides concrete answers to these questions.

large language model, machine learning, natural language, (19 more...)

Neural Information Processing Systems

Feb-17-2026, 06:30:27 GMT

Conferences PDF

Country:
- Asia > Middle East
  - Jordan (0.04)
- North America > United States
  - California > Santa Clara County
    - Stanford (0.04)
  - Minnesota (0.04)

Genre:
- Research Report
  - Experimental Study (0.93)
  - New Finding (1.00)

Technology:
- Information Technology > Artificial Intelligence
  - Machine Learning
    - Neural Networks > Deep Learning (1.00)
    - Statistical Learning (0.87)
  - Natural Language > Large Language Model (0.67)
  - Representation & Reasoning > Optimization (1.00)