attention head
Country:
- Europe > Germany > Saarland (0.04)
- Asia > Philippines (0.04)
- North America > Canada > Ontario > Toronto (0.04)
- (8 more...)
Genre:
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (0.92)
Industry:
- Leisure & Entertainment (0.45)
- Information Technology (0.45)
- Education (0.45)
Technology:
Knowledge Circuits in Pretrained Transformers
In this paper, we delve into the computation graph of the language model to uncover the knowledge circuits that are instrumental in articulating specific knowledge.
Genre:
- Research Report > Experimental Study (0.93)
- Research Report > New Finding (0.68)
Industry:
- Information Technology (0.46)
- Education (0.46)
Technology:
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.69)
Genre:
- Research Report > Experimental Study (0.92)
- Research Report > New Finding (0.67)
Technology:
Country:
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- North America > United States > California (0.14)
- Europe > Italy > Tuscany > Florence (0.04)
- (12 more...)
Technology:
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing
Due to their size, the capability of these networks has increased tremendously, but this has come at the cost of a significant increase in necessary compute.
Country:
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- North America > Dominican Republic (0.04)
- Europe > Netherlands > North Holland > Amsterdam (0.04)
- (5 more...)
Technology:
UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation
Large language models (LLMs) have demonstrated impressive capabilities in various tasks using the in-context learning (ICL) paradigm.
Country:
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- Asia > Singapore (0.04)
- North America > Canada > Ontario > Toronto (0.04)
- (11 more...)
Genre:
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
Technology:
Tailoring Self-Attention for Graph via Rooted Subtrees
However, local attention limits the receptive field to one-hop neighbors.
Country:
- North America > United States > California > Los Angeles County > Long Beach (0.14)
- Europe > Austria (0.04)
- Asia > China > Shanghai > Shanghai (0.04)
- (8 more...)
Technology:
Country:
- North America > United States > Indiana (0.04)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.04)
- Asia > Japan > Honshū > Tōhoku > Fukushima Prefecture > Fukushima (0.04)
Technology:
Country:
- North America > United States (0.14)
- Asia > Middle East > Jordan (0.04)
- Asia > China (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
Genre:
- Research Report > New Finding (0.92)
- Overview (0.92)