How regulatory pressure is reshaping big data as we know it

It should come as no surprise to anyone monitoring the burgeoning big data ecosystem that the increasing quantity of data being generated, the majority of which is unstructured--combined with the growing number of external data sources and quotidian nature of data breaches--has led to today's hyper-sensitive regulatory environment. It was only natural that the ability to automate the processing, production, analysis, and management of big data via cognitive computing dominated the epoch in which real-time transactions (in the cloud, via mobile technologies and ecommerce) became the norm. The rapid dissemination of personally identifiable information (PII), the expansion of its definitions, and the inherent incongruities between regulations were similarly logical conclusions of the same vector in which automation and decision-support were esteemed. But when these same big data developments led to issues of interpretability and "explainability," and when people or intelligent systems simply relied on quantifiable algorithmic outputs with limited understanding of their biases or the reasons behind them, intervention--in the form of regulatory mandates and penalties--also quite naturally arose. Some are international in scope and jurisdiction, such as the recently implemented General Data Protection Regulation (GDPR).