In 2002, Target hired statistician Andrew Pole. His job was to use predictive analytics -- a form of statistics that makes predictions by observing data trends -- to help the retail giant market certain products to certain groups of people. Along those lines, Pole's first task was to identify pregnant women -- specifically women in their second trimester. As Target's marketing team explained to him, new parents are extremely valuable customers whose brand loyalty tends to change when they have kids because they purchase things they probably weren't purchasing before -- like diapers, formula, baby clothes, etc. New parents also tend to be physically exhausted and therefore more prone to do all of their shopping at one place.

Today's digital economy is powered by data, which is produced in abundance by both individuals and enterprises and stored in vast data centres, according to GlobalData, a leading data and analytics company. The company's latest report, 'Big data – Thematic Research', details how several prominent business people and a number of leading publications have described data as the new oil, capable of generating significant value if used in the right way. Many big data vendors have had to contend with a growing market perception that data governance, security, and management have taken a back seat to accessibility and speed. In response, most companies are now accepting the challenge and openly prioritising data governance. This is expected to result in multiple disparate solutions being replaced by single data management platforms, leading to efficient scalability, collection, and distribution of data.

Though we cannot tell what the future holds for us, we can make predictions based on trends. JT Kostman Ph.D – A global cyber crime pandemic ($6T annually) and an ever-expanding alphabet soup of data privacy/protection legislation will increasingly require Data Scientists to accept dual responsibilities as data fiduciaries. As the volume, velocity, variety, virality, and viciousness of cyber attacks inexorably increases, AI solutions will increasingly become the only way to compensate for the projected shortfall of 1.8 million cyber security professionals otherwise needed to combat increasingly sophisticated and determined adversaries – and keep corporate executives out of court. Cassie Kozurkov – We can expect to see improvements in tools for data science as more user experience designers take an interest in the data scientist as user. Image data will grow in importance as the camera becomes more than a way to capture memories, but evolves towards a more natural way for users to interact with apps.

In 2002, Target hired statistician Andrew Pole. His job was to use predictive analytics -- a form of statistics that makes predictions by observing data trends -- to help the retail giant market certain products to certain groups of people. Along those lines, Pole's first task was to identify pregnant women -- specifically women in their second trimester. As Target's marketing team explained to him, new parents are extremely valuable customers whose brand loyalty tends to change when they have kids because they purchase things they probably weren't purchasing before -- like diapers, formula, baby clothes, etc. New parents also tend to be physically exhausted and therefore more prone to do all of their shopping at one place.

Check out the Strata Data and Artificial Intelligence conference series, which cover the topics and key issues discussed in this post. Whether you're a business leader or a practitioner, here are key data trends to watch and explore in the months ahead. In a recent O'Reilly survey, we found that the skills gap remains one of the key challenges holding back the adoption of machine learning. The demand for data skills ("the sexiest job of the 21st century") hasn't dissipated. LinkedIn recently found that demand for data scientists in the US is "off the charts," and our survey indicated that the demand for data scientists and data engineers is strong not just in the US but globally.

Extracting the underlying trend signal is a crucial step to facilitate time series analysis like forecasting and anomaly detection. Besides noise signal, time series can contain not only outliers but also abrupt trend changes in real-world scenarios. To deal with these challenges, we propose a robust trend filtering algorithm based on robust statistics and sparse learning. Specifically, we adopt the Huber loss to suppress outliers, and utilize a combination of the first order and second order difference on the trend component as regularization to capture both slow and abrupt trend changes. Furthermore, an efficient method is designed to solve the proposed robust trend filtering based on majorization minimization (MM) and alternative direction method of multipliers (ADMM). We compared our proposed robust trend filter with other nine state-of-the-art trend filtering algorithms on both synthetic and real-world datasets. The experiments demonstrate that our algorithm outperforms existing methods.

It is very difficult for organisations to discern which tools will bring them the most benefit in the year ahead, and which issues they need to plan for, such is the volume of technological choices available, data trends is no exception to this. New technological developments provide the platform for the next generation of innovation, as we've seen with the evolution of'Big Data' into advanced analytics, machine learning and artificial intelligence. How can businesses navigate this increasingly-complex data landscape to make the wisest investments? Here is our guide to the top seven data trends that should be on every organisation's radar for the year ahead. We recently conducted research which found that 85% of respondents said they already had some of their data infrastructure in the cloud, and other surveys of IT executives reveal that many are planning to increase their investments in Software as a Service (SaaS) and cloud tools.

Did you know that we also offer Custom Research? Visit our Custom Research page to learn more and schedule a meeting with our Custom Research Manager.

Building an operating system for data is the foundation needed for any industry today. These could range from a health care organization crafting a strategy to comply with General Data Protection Regulation (GDPR) and Health Insurance Portability and Accountability Act (HIPAA) policies, to an airline using artificial intelligence (AI) to pinpoint maintenance issues sooner and get passengers to their destinations on time. As we move further into the new year, companies may need to harness greater amounts of their data for competition and innovation. This will not only help to solve the challenges surrounding dark data and upcoming data regulations but will also open the door to uncovering new ways to innovate with data and AI. As more organizations take hold of their data with cloud technology, we can expect these five trends to continue to change the way we view the potential of cloud and AI in 2018.

Wearables are ubiquitous; everyone worth their salt is jumping on the Internet of Things (IoT) bandwagon; and Augmented Reality (AR) is here to stay. These are the current favorite trends in the industry; and based on what we've seen in the recent years, we think it'll be mostly how businesses apply these trends that will drive the growth of these technologies, not the end users. And here is why - the data generated by these three technologies is incredibly voluminous, we all know that. It might not be of much use to an individual user, but in large volumes can be incredibly useful to businesses. If we can handle its relentless onslaught, this data can especially provide us deep real-time analytical insights that can help brands understand their consumers far better and make data-driven decisions. Data-converging platforms that will converge the influx of all connected data will rise in 2017, fuelled by the demand to tame this massive data.