Kubernetes Volume Controller (KVC): Data Management Tailored for Machine Learning Workloads in Kubernetes - Intel AI

Data is an important component in ML workloads and pipelines. Typically, data scientists and ML practitioners handle the data using existing primitives available through a scheduling system such as Kubernetes. However, the users still need to keep track of the data as well as the relationship between data and the primitives used. Data from multiple sources might be required to run their workloads and pipelines. In some cases (e.g., hyperparameter tuning), the data might need to be replicated or made available in some of the compute nodes in a cluster.