This book will help demonstrate how to use Kubeflow components, deploy them in GCP, and serve them in production using real-time data prediction. With Kubeflow KFserving, we'll look at serving techniques, build a computer vision-based user interface in streamlit, and then deploy it to the Google cloud platforms, Kubernetes and Heroku. Next, we also explore how to build Explainable AI for determining fairness and biasness with a What-if tool. Backed with various use-cases, we will learn how to put machine learning into production, including training and serving.

One thing that we can do to mitigate those risks is to draw inspiration from some of the biggest companies in the world that are deploying machine learning at scale. Today, we would like to discuss some of the reference architectures used by AI powerhouses like Google, Facebook, LinkedIn, and Uber to enable their machine learning pipelines. One of the best-known efforts in this area, Uber's Michelangelo is the runtime powering hundreds of machine learning workflows at Uber. From experimentation to model serving, Michelangelo combines mainstream technologies to automate the lifecycle of machine learning applications. The architecture behind Michelangelo uses a modern but complex stack based on technologies such as HDFS, Spark, Samza, Cassandra, MLLib, XGBoost, and TensorFlow.

Ever since Google has publicised Tensorflow, its application in Deep Learning has been increasing tremendously. It is used even more in research and production for authoring ML algorithms. Though it is flexible, it does not provide an end-to-end production system. On the other hand, Sibyl has end-to-end facilities but lacks flexibility. Google then came up with Tensorflow Extended(TFX) idea as a production-scaled machine learning platform on Tensorflow, taking advantage of both Tensorflow and Sibyl frameworks.

Software Engineering, as a discipline, has matured over the past 5 decades. The modern world heavily depends on it, so the increased maturity of Software Engineering was an eventuality. Practices like testing and reliable technologies help make Software Engineering reliable enough to build industries upon. Meanwhile, Machine Learning (ML) has also grown over the past 2 decades. ML is used more and more for research, experimentation and production workloads. But ML Engineering, as a discipline, has not widely matured as much as its Software Engineering ancestor. Can we take what we have learned and help the nascent field of applied ML evolve into ML Engineering the way Programming evolved into Software Engineering? In this article we will give a whirlwind tour of Sibyl and TensorFlow Extended (TFX), two successive end-to-end (E2E) ML platforms at Alphabet. We will share the lessons learned from over a decade of applied ML built on these platforms, explain both their similarities and their differences, and expand on the shifts (both mental and technical) that helped us on our journey.

We discuss the use of Machine Learning pipeline architectures for implementing production ML applications, and in particular we review Google's experience with TensorFlow Extended (TFX). An ML application in production must address all of the issues of modern software development methodology, as well as issues unique to ML and data science. Most of the focus in the ML community is on research, which is exciting and important. Equally important however is bringing that research to production applications to solve real-world problems, but the issues and approaches for doing that are often poorly understood. An ML application in production must address all of the issues of modern software development methodology, as well as issues unique to ML and data science.

There is more to building a sustainable Deep Learning solution than what is provided by Deep Learning frameworks like TensorFlow and PyTorch. These frameworks are good enough for research, but they don't take into account the problems that crop up with production deployment. I've written previously about technical debt and the need from more adaptive biological like architectures. To support a viable business using Deep Learning, you absolutely need an architecture that supports sustainable improvement in the presence of frequent and unexpected changes in the environment. Current Deep Learning framework only provide a single part of a complete solution.

Creating and maintaining a platform for reliably producing and deploying machine learning models requires careful orchestration of many components---a learner for generating models based on training data, modules for analyzing and validating both data as well as models, and finally infrastructure for serving models in production. This becomes particularly challenging when data changes over time and fresh models need to be produced continuously. Unfortunately, such orchestration is often done ad hoc using glue code and custom scripts developed by individual teams for specific use cases, leading to duplicated effort and fragile systems with high technical debt. We present the anatomy of a general-purpose machine learning platform and one implementation of such a platform at Google. By integrating the aforementioned components into one platform, we were able to standardize the components, simplify the platform configuration, and reduce the time to production from the order of months to weeks, while providing platform stability that minimizes service disruptions.

What world-class looks like in online product and service development has been undergoing quite the revolution over the last few years. The series of papers we've been looking at recently can help you to understand where the bar is (it will have moved on again by the time most companies get there of course!). Just to be clear here, I'm not saying you necessarily need to build your own versions of all of these platforms. It's more about embracing their use as part of your everyday practices, and if you can do that by renting then for many organisations that's going to be the best choice. The code that implements your machine learning model is only a tiny part of what goes into using machine learning in production systems.