Deep neural networks (DNNs) continue to grow rapidly in size, making them infeasible to train on a single device. Pipeline parallelism is commonly used in existing DNN systems to support large-scale DNN training by partitioning a DNN into multiple stages, which concurrently perform DNN training for different micro-batches in a pipeline fashion. However, existing pipeline-parallel approaches only consider sequential pipeline stages and thus ignore the topology of a DNN, resulting in missed model-parallel opportunities. This paper presents graph pipeline parallelism (GPP), a new pipeline-parallel scheme that partitions a DNN into pipeline stages whose dependencies are identified by a directed acyclic graph. GPP generalizes existing sequential pipeline parallelism and preserves the inherent topology of a DNN to enable concurrent execution of computationally-independent operators, resulting in reduced memory requirement and improved GPU performance. In addition, we develop GraphPipe, a distributed system that exploits GPP strategies to enable performant and scalable DNN training. GraphPipe partitions a DNN into a graph of stages, optimizes micro-batch schedules for these stages, and parallelizes DNN training using the discovered GPP strategies. Evaluation on a variety of DNNs shows that GraphPipe outperforms existing pipeline-parallel systems such as PipeDream and Piper by up to 1.6X. GraphPipe also reduces the search time by 9-21X compared to PipeDream and Piper.

Oracle has released an open-source tool called Graphpipe to facilitate the deployment of machine learning models. Graphpipe will simplify the use of machine learning for mobile apps and IoT devices, end user services, and internal corporate functions. Graphpipe could obviate the need for developers to create custom APIs to deploy AI models or to tailor their work to a specific framework. "Graphpipe is an attempt to standardize the protocol by which you speak to a remotely deployed machine learning model, and it includes some reference servers that allow you to deploy machine learning models from existing frameworks very easily in an efficient way," says Oracle's Vish Abrams. Although developers have several framework choices to build AI models, fewer options exist for how to serve or deploy AI models, Abrams says.

It has become increasingly apparent that there is a tremendous opportunity when it comes to infusing artificial intelligence (AI) and blockchain platforms. AI, otherwise known as machine learning in some circle, as the ability to take blockchain operations and effectiveness to the next level. Alternatively, blockchain technology can also work in the opposite direction, allowing cutting-edge tech leaders to understand AI in greater depths and essentially create more efficient and smarter intelligence platforms. By decentralizing the market, blockchain and AI will work together to make transactions more secure, while simultaneously generating large amounts of new revenues. Already multi-billion dollar industries on their own, blockchain and AI are poised to benefit on a high scale financially as the two become more intertwined.

"We created GraphPipe to solve these three challenges. It provides a standard, high-performance protocol for transmitting tensor data over the network, along with simple implementations of clients and servers that make deploying and querying machine learning models from any framework a breeze.

Oracle, a company not exactly known for having the best relationship with the open source community, is releasing a new open source tool today called Graphpipe, which is designed to simplify and standardize the deployment of machine learning models. The tool consists of a set of libraries and tools for following the standard. Vish Abrams, whose background includes helping develop OpenStack at NASA and later helping launch Nebula, an OpenStack startup in 2011, is leading the project. He says as his team dug into the machine learning workflow, they found a gap. While teams spend lots of energy developing a machine learning model, it's hard to actually deploy the model for customers to use.