Intel announced Nauta, an open source deep learning project based on Kubernetes. The project comes with select open source components and Intel-developed custom applications, tools, and scripts for building deep learning models. According to Intel, Nauta provides a multi-user, distributed computing environment for running deep learning model training experiments on systems based on Intel Xeon processor. The software foundation for the distributed platform is built on Kubernetes – industry's leading container orchestration engine. Mainstream deep learning tools and frameworks such as TensorFlow, TensorBoard and Jupyter Hub are tightly integrated with the platform.
Intel has released a new platform dubbed Nauta that lets data scientists and developers use Kubernetes and Docker to conduct distributed deep learning (DL) at scale. Its client software, open sourced today under an Apache 2.0 licence, can run on Ubuntu 16.04, Red Hat 7.5, MacOS High Sierra and Windows 10. It supports both batch and streaming inference, its GitHub repo shows. Streaming inference entails deploying a model on a system and processing singular data as it is received). Hugely popular orchestration engine Kubernetes, originally developed by Google, makes it easier to build and deploy container-based applications.
Intel today unveiled Nauta, a new distributed computing framework for running deep learning models in Kubernetes and Docker-based environments. The chip giant says Nauta will make it easier for data scientists to develop, train, and deploy deep learning workloads on large clusters, "without all the systems overhead and scripting needed with standard container environments." Deep learning has become one of the hottest areas of machine learning, thanks to its capability to train highly accurate predictive models in areas like computer vision and natural language processing. The technique, however, generally requires huge amounts of data and large amounts of computing horsepower, which today's enterprises typically want to manage using containers. Getting all these pieces to work together – Kubernetes, Docker, and deep learning frameworks – is complex and hard.
Intel today announced the open source release of Nauta, a platform for deep learning distributed across multiple servers using Kubernetes or Docker. The platform can operate with a number of popular machine learning frameworks such as MXNet, TensorFlow, and PyTorch, and uses processing systems that can work with a cluster of Intel's Xeon CPUs. Results of deep learning experiments conducted with Nauta can be seen using TensorBoard, command line code, or a Nauta web user interface. "Nauta is an enterprise-grade stack for teams who need to run DL workloads to train models that will be deployed in production. With Nauta, users can define and schedule containerized deep learning experiments using Kubernetes on single or multiple worker nodes, and check the status and results of those experiments to further adjust and run additional experiments, or prepare the trained model for deployment," according to a blog post announcing the news today.
Many data scientists today find it burdensome to manually execute all of the steps in a machine learning workflow. Moving and transforming data, training models, then promoting them into production – all of it requires the data scientist's close attention. But now an open source project called Kubeflow promises to eliminate much of that busywork by automating machine learning workflows atop Kubernetes clusters. Google initially created Kubeflow to manage its internal machine learning pipelines written in Tensorflow and executed atop Kubernetes, and released it as an open source project in late 2017. Since then, the Kubeflow community has integrated the software with a handful of additional machine learning and deep learning frameworks, including MXnet, PyTorch, Caffe2, and Nvidia TensorRT, as well as Jupyter notebooks and MPI, the parallel computing framework used in high performance computing (HPC) clusters.