Today, we are excited to announce the beta release of TensorFlow in the Mesosphere DC/OS Service Catalog. Using a single command, you can now deploy distributed TensorFlow on any bare-metal, virtual, or public cloud infrastructure. As with other packages available for DC/OS, the new TensorFlow package also includes the ability to use GPUs to accelerate your machine learning and deep learning applications. In the race to leverage deep learning capabilities, data scientists specializing in deep learning are highly sought after. An efficient data science infrastructure allows you to attract the best data scientists and get the best work out of them, which gives your business a strategic advantage over competitors.
If you are reading this you've probably already started your journey into deep learning. If you are new to this field, in simple terms deep learning is an add-on to develop human-like computers to solve real-world problems with its special brain-like architectures called artificial neural networks. To help develop these architectures, tech giants like Google, Facebook and Uber have released various frameworks for the Python deep learning environment, making it easier for to learn, build and train diversified neural networks. In this article, we'll take a look at two popular frameworks and compare them: PyTorch vs. TensorFlow. TensorFlow is open source deep learning framework created by developers at Google and released in 2015.
Yahoo has announced TensorFlowOnSpark, its latest open source framework for distributed deep learning on big data clusters. Deep learning (DL) has evolved significantly in recent years. At Yahoo, we've found that in order to gain insight from massive amounts of data, we need to deploy distributed deep learning. Existing DL frameworks often require us to set up separate clusters for deep learning, forcing us to create multiple programs for a machine learning pipeline (see Figure 1 below). Having separate clusters requires us to transfer large datasets between them, introducing unwanted system complexity and end-to-end learning latency.
In the context of machine learning, tensor refers to the multidimensional array used in the mathematical models that describe neural networks. In other words, a tensor is usually a higher-dimension generalization of a matrix or a vector. Through a simple notation that uses a rank to show the number of dimensions, tensors allow the representation of complex n-dimensional vectors and hyper-shapes as n-dimensional arrays. Tensors have two properties: a datatype and a shape. TensorFlow is an open source deep learning framework that was released in late 2015 under the Apache 2.0 license.