Data Quality Evolution with Big Data and Machine Learning Transforming Data with Intelligence

When big data is combined with machine learning, enterprises must be alert to new data quality issues. IT departments have been struggling with data quality issues for decades, and satisfactory solutions have been found for ensuring quality in structured data warehouses. However, big data solutions, unstructured data, and machine learning are creating new types of quality issues that must be addressed. Big data affects quality because its defining features of volume, variety, and velocity make verification difficult. The elusive "fourth V," the veracity component (concerning data reliability), is challenging due to the large number of data sources that might be brought together, each of which might be subject to different quality problems.