On the Role of Attention Masks and LayerNorm in Transformers

Nov-14-2025, 06:02:16 GMT–Neural Information Processing Systems

Self-attention is the key mechanism of transformers, which are the essential building blocks of modern foundation models.

large language model, layernorm, machine learning, (22 more...)

Neural Information Processing Systems

Nov-14-2025, 06:02:16 GMT

Conferences PDF

Country:
- Asia > Middle East
  - Jordan (0.04)
- Europe > Germany
  - Bavaria > Upper Bavaria > Munich (0.04)
- North America > United States
  - Massachusetts > Middlesex County > Cambridge (0.04)
- South America > Chile
  - Santiago Metropolitan Region > Santiago Province > Santiago (0.04)

Genre:
- Research Report
  - Experimental Study (1.00)
  - New Finding (0.93)

Industry:
- Government > Regional Government (0.46)

Technology:
- Information Technology > Artificial Intelligence
  - Machine Learning > Neural Networks
    - Deep Learning (1.00)
  - Natural Language > Large Language Model (1.00)
  - Representation & Reasoning (1.00)
  - Vision (0.67)