Deep learning, one of the most fascinating subjects in computer science, has spawned a slew of machine learning frameworks and libraries, sparking community discussions about platforms like PyTorch vs TensorFlow. Currently, the most prominent frameworks are PyTorch and TensorFlow, which were created by Facebook and Google, respectively. Both of these frameworks are open-source libraries for machine learning that are widely utilised in commercial and academic research. They're also distinct enough that you'll want to think about the framework you'll use before getting started. Why is there a debate between PyTorch and TensorFlow in the machine learning community?
The best way to compare two frameworks is to code something up in both of them. I've written a companion jupyter notebook for this post and you can get it here. All code will be provided in the post too. First, let's code a simple approximator for the following function in both frameworks: We will try to find unknown parameter phi given data x and function values f(x). Yes, using stochastic gradient descent for this is an overkill and analytical solution may be found easily, but this problem will serve our purpose well as a simple example.
If you are reading this you've probably already started your journey into deep learning. If you are new to this field, in simple terms deep learning is an add-on to develop human-like computers to solve real-world problems with its special brain-like architectures called artificial neural networks. To help develop these architectures, tech giants like Google, Facebook and Uber have released various frameworks for the Python deep learning environment, making it easier for to learn, build and train diversified neural networks. In this article, we'll take a look at two popular frameworks and compare them: PyTorch vs. TensorFlow. TensorFlow is open source deep learning framework created by developers at Google and released in 2015.
Recall RNNs: with static graphs, the input sequence length will stay constant. Here we introduce datasets module which contains wrappers for popular datasets used to benchmark deep learning architectures. There are large amounts of ready to use modules in torch.nn Notice how PyTorch uses object oriented approach to define basic building blocks and give us some'rails' to move on while providing ability to extend functionality via subclassing. Here we will use tf.layers and tf.contrib.learn The code follows the official tutorial on tf.layers: So, both TensorFlow and PyTorch provide useful abstractions to reduce amounts of boilerplate code and speed up model development.
This is a guide to the main differences I've found between PyTorch and TensorFlow. This post is intended to be useful for anyone considering starting a new project or making the switch from one deep learning framework to another. The focus is on programmability and flexibility when setting up the components of the training and deployment deep learning stack. I won't go into performance (speed / memory usage) trade-offs. PyTorch is better for rapid prototyping in research, for hobbyists and for small scale projects.