Introduction to Information Theory and Why You Should Care - Recast.AI Blog

Let's talk about information – what is it? What are the limits on the communication of information? We'll try to answer these questions by looking at the revolutionary work of one man – Claude E. Shannon, and the men and women who followed in his footsteps. Most importantly, we'll try to answer the question'is this important for machine learning and why?'. I hope that by the time you finish reading this post you too will be convinced that information theory can be very beneficial to anybody who's interested in developing machine learning systems. Let's start by discussing the general way the word "information" was understood before Shannon's work in 1948. Then we will be ready to understand the revolutionary nature of this work.