Data Validation in Machine Learning is Imperative, Not Optional - KDnuggets

Operationalizing a Machine Learning (ML) model in production needs a lot more than just creating and validating models like in academia or research. The ML application in production can be a pipeline with multiple components running consecutively as shown in Fig 1. Before we reach model training in the pipeline, there are various components like Data Ingestion, Data versioning, Data validation, and Data pre-processing that need to be executed. Data validation means checking the accuracy and quality of source data before training a new model version. It ensures that anomalies that are infrequent or manifested in incremental data are not silently ignored.