Understand TensorFlow by mimicking its API from scratchTheory

Over the years it has become one of the most popular libraries for deep learning. The goal of this post is to build an intuition and understanding for how deep learning libraries work under the hood, specifically TensorFlow. To achieve this goal, we will mimic its API and implement its core building blocks from scratch. This has the neat little side effect that, by the end of this post, you will be able to use TensorFlow with confidence, because you'll have a deep conceptual understanding of the inner workings. You will also gain further understanding of things like variables, tensors, sessions or operations. So let's get started, shall we? Note: If you are familiar with the basics of TensorFlow including how computational graphs work, you may skip the theory and jump straight to the implementation part. TensorFlow is a framework composed of two core building blocks -- a library for defining computational graphs and a runtime for executing such graphs on a variety of different hardware. A computational graph has many advantages but more on that in just a moment. Now the question you might ask yourself is, what exactly is a computational graph?