Typically, the development of GPU kernels is a laborious process. However, if the algorithms can be expressed using combinations of high-level operators then it should be possible to generate the GPU kernel. This is what CCT is designed to do. An offshoot of CCT is the Operator Vectorization Library (OVL). OVL is a python library that does the same a CCT but for TensorFlow framework.

Typically, the development of GPU kernels is a laborious process. However, if the algorithms can be expressed using combinations of high-level operators then it should be possible to generate the GPU kernel. This is what CCT is designed to do. An offshoot of CCT is the Operator Vectorization Library (OVL). OVL is a python library that does the same a CCT but for TensorFlow framework.

Here's the deal, you've probably never heard of SystemML, but you definitely need to know what it is. Not only will SystemML make you look awesome because machine learning is the hot topic right now, but it will also save you a lot of time and trouble. As a new data scientist I am constantly having to spend my time learning new technologies--most of which don't work very well. Here's the thing: SystemML actually does work very well. Because it only recently became open source, it's difficult to find material on how to get started, but that's quickly changing.

This tutorial will get you set up and running SystemML on the Spark Shell like a star. But first, to refresh your memory, let me remind you that I am on a quest to create a life-changing app! I am new to the world of data science and am currently tackling the challenge of building an app using Apache SystemML and Apache Spark one step at a time. If you haven't already, make sure to check out my previous tutorials, which start here. So far we've daydreamed about delightful data, complained about how hard it is to find good data, found good data, learned how to write Scala and NOW we will learn how to access SystemML from the Spark Shell.

IBM has fulfilled its promise to open-source SystemML, a machine learning system that's now been accepted as an Apache Incubator project. It's a significant milestone for SystemML, which is already used to power IBM's BigInsights data analytics platform. The Apache Incubator program is a kind of stepping stone on the way to becoming a full project under The Apache Software Foundation, where developers ensure code donations adhere to the ASF's guidelines and that the community follows its principles. The SystemML technology emerged from IBM's development of Watson, and integrates closely with another Apache project, Spark. SystemML helps Watson to keep up to date by providing a language that directly exposes the capabilities of the artificial intelligence so data scientists can harvest it.

The reputation of SystemML is on the rise. This flexible machine learning system scales automatically to Spark and Hadoop clusters and offers faster analysis on fewer nodes -- with substantial improvements in accuracy. In this presentation from Spark Summit in June 2016, researcher Fred Reiss spells out use cases for custom ML algorithms -- across the spectrum from auto manufacturing to the Watson Health initiative.

Machine learning explores the study and construction of algorithms that learn and make predictions based on data. In the field of machine learning, data scientists, who specialize in analyzing data, are responsible for writing and modifying such algorithms. Initially, a data scientist writes an algorithm based on a set of data features. This is generally an iterative process in which the data scientist explores different algorithms for predictive purpose. In this process, the amount of data and the number of features chosen for analysis may change.