AITopics | optimizer and numerical stability

Collaborating Authors

optimizer and numerical stability

Information about AI from the News, Publications, and Conferences

Automatic Classification – Tagging and Summarization – Customizable Filtering and Analysis

If you are looking for an answer to the question What is Artificial Intelligence? and you only have a minute, then here's the definition the Association for the Advancement of Artificial Intelligence offers on its home page: "the scientific understanding of the mechanisms underlying thought and intelligent behavior and their embodiment in machines."

However, if you are fortunate enough to have more than a minute, then please get ready to embark upon an exciting journey exploring AI (but beware, it could last a lifetime) …

[D] Optimizers and numerical stability for Bayesian deep nets

#artificialintelligenceNov-24-2020, 06:05:46 GMT

I've been experimenting with flipout layers with tensorflow and tensorflow probability, but I keep having numerical stability issues in late epochs using negative log likelihood losses with adam. It also seems like the network has a lot of difficulty with learning scale parameters, (e.g. the standard deviation if the probability model is normal) particularly when the n is smallest, and returns a lot of samples far, far outside the range of the training data.

bayesian deep net, optimizer and numerical stability

#artificialintelligence

Industry: Media > News (0.40)

Technology:

Information Technology > Artificial Intelligence > Machine Learning (1.00)
Information Technology > Communications > Social Media (0.76)

Add feedback