Dark analytics: Illuminating opportunities hidden within unstructured data

Across enterprises, ever-expanding stores of data remain unstructured and unanalyzed. Few organizations have been able to explore nontraditional data sources such as image, audio, and video files; the torrent of machine and sensor information generated by the Internet of Things; and the enormous troves of raw data found in the unexplored recesses of the "deep web." However, recent advances in computer vision, pattern recognition, and cognitive analytics are making it possible for companies to shine a light on these untapped sources and derive insights that lead to better experiences and decision making across the business. In this age of technology-driven enlightenment, data is our competitive currency. Buried within raw information generated in mind-boggling volumes by transactional systems, social media, search engines, and countless other technologies are critical strategic, customer, and operational insights that, once illuminated by analytics, can validate or clarify assumptions, inform decision making, and help chart new paths to the future. Until recently, taking a passive, backward-looking approach to data and analytics was standard practice. With the ultimate goal of "generating a report," organizations frequently applied analytics capabilities to limited samples of structured data siloed within a specific system or company function. Moreover, nagging quality issues with master data, lack of user sophistication, and the inability to bring together data from across enterprise systems often colluded to produce insights that were at best limited in scope and, at worst, misleading.