Goto

Collaborating Authors

 know information theory concept


Must know Information Theory concepts in Deep Learning (AI)

#artificialintelligence

In the early 20th century, scientists and engineers were struggling with the question: "How to quantify the information? Is there a analytical way or a mathematical measure that can tell us about the information content?". It is not difficult to tell that the second sentence gives us more information since it also tells that Bruno is "big" and "brown" in addition to being a "dog". How can we quantify the difference between two sentences? Can we have a mathematical measure that tells us how much more information second sentence have as compared to the first?