ModelDB: A System for Managing Machine Learning Models

Moreover, we found that data scientists usually had an ML environment of choice (e.g., scikit-learn, R, spark.ml), As a result, we implemented native logging libraries for different ML environments that would capture models built by a data scientist along with pre-processing operations performed on the data (e.g., one-hot-encoding, scaling). As of now, we have written logging libraries for scikit-learn and spark.ml. Libraries for different ML environments implement a ModelDB thrift interface that is used to communicate with the backend.