Spotify Open-Sources Terraform Module for Kubeflow ML Pipelines

Spotify has open-sourced their Terraform module for running machine-learning pipeline software Kubeflow on Google Kubernetes Engine (GKE). By switching their in-house ML platform to Kubeflow, Spotify engineers have achieved faster time to production and are producing 7x more experiments than on the previous platform. In a recent blog post, Spotify's product manager Josh Baer and ML engineer Samuel Ngahane described Spotify's "Paved Road" for machine learning: "an opinionated set of products and configurations to deploy an end-to-end machine learning solution using our recommended infrastructure." By adopting these standards, Spotify's machine learning engineers no longer need to build or maintain infrastructure and instead can focus on their ML experiments. Since launching the platform in mid-2019, about 100 internal users have adopted it and run up to 18,000 experiments.