arxivpreprint
Country:
- North America > United States (0.05)
- Africa > Ethiopia (0.04)
- Europe > United Kingdom (0.04)
Country:
- North America > United States (0.05)
- Africa > Ethiopia (0.05)
- Europe > United Kingdom (0.04)
Country:
- Asia > Japan > Honshū > Chūbu > Ishikawa Prefecture > Kanazawa (0.04)
- Asia > Middle East > Jordan (0.04)
- North America > Canada (0.04)
- Asia > Japan > Honshū > Chūbu > Nagano Prefecture > Nagano (0.04)
Technology:
Country: North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.04)
Breaking the Activation Function Bottleneck through Adaptive Parameterization
Sebastian Flennerhag, Hujun Yin, John Keane, Mark Elliot
Adaptive parameterization is a means of increasing this flexibility and thereby increasing the model's capacity to learn non-linear patterns. We focus on the feed-forward layer, f(x):= φ(W x+b),for some activation functionφ: R 7 R. Define the pre-activation layer as a = A(x):= Wx+band denote byg(a):= φ(a)/athe activation effect ofφgivena, where divisioniselement-wise.
Technology: Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.62)
Technology: Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.67)
Country:
- North America > United States > Massachusetts > Suffolk County > Boston (0.04)
- North America > Canada > Quebec > Montreal (0.04)
- Europe > United Kingdom > England > Tyne and Wear > Newcastle (0.04)
- (3 more...)
Country:
- North America > Canada > Quebec > Montreal (0.04)
- Europe > United Kingdom > England (0.04)
Technology:
Country:
- North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.04)
- Europe > Germany > Bavaria > Upper Bavaria > Munich (0.04)
Technology: