With companies such as Amazon Web Services offering AI on tap -- including image recognition and natural language understanding -- an increasing number of machine learning powered services will be finding their way to the consumer. Those involved in marketing and retail might have already seen offerings filter through, with the following pointing the way the AI winds are blowing. In the world of AI, Watson is one of the best-known names out there. Built by IBM, this computer system is able to learn from massive amounts of unstructured data, while also being able to answer natural language questions. Already having made an impact in the healthcare industry, Watson is bringing its AI capabilities to a number of other sectors too.

In Personalization at Amazon, we use neural networks to generate personalized product recommendations for our customers. Amazon's product catalog is huge compared to the number of products that a customer has purchased, making our datasets extremely sparse. And with hundreds of millions of customers and products, our neural network models often have to be distributed across multiple GPUs to meet space and time constraints. For this reason, we have created and open-sourced DSSTNE, the Deep Scalable Sparse Tensor Neural Engine, which runs entirely on the GPU. We use DSSTNE to train neural networks and generate recommendations that power various personalized experiences on the retail website and Amazon devices.

Amazon discreetly released a library called DSSTNE on GitHub under an open-source Apache license and made an entry into the world of open-source software for deep learning. DSSTNE (pronounced "Destiny") is an open source software library for training and deploying deep neural networks using GPUs. Amazon engineers built DSSTNE to solve deep learning problems at Amazon's scale. DSSTNE is built for production deployment of real-world deep learning applications, emphasizing speed and scale over experimental flexibility. "DSSTNE's network definition language is much simpler than Caffe's, as it would require only 33 lines of code to express the popular AlexNet image recognition model, whereas Caffe's language requires over 300 lines of code," Amazon wrote on the FAQ page.

Among technology categories creating sweeping change right now, cloud computing and Big Data analytics dominate the headlines, and open source platforms are making a difference in these categories. However, one of the biggest open source stories of the year surrounds newly contributed projects in the field of artifical intelligence and the closely related field of machine learning. Some of the biggest tech companies are helping to drive the trend. Google has open sourced a program called TensorFlow. It's based on the same internal toolset that Google has spent years developing to support its AI software.

Following examples set by the likes of Google and others, Amazon has made its Deep Scalable Sparse Tensor Network Engine (DSSTNE) generally available to researchers, developers and everyone else. The engine, which is used to provide product recommendations to Amazon shoppers -- usually under the "You may also be interested in" -- is now available on Github. The package includes examples, instructions for setup, FAQs, User guide and holds a business-friendly Apache 2.0 license. We are releasing DSSTNE as open source software so that the promise of deep learning can extend beyond speech and language understanding and object recognition to other areas such as search and recommendations. We hope that researchers around the world can collaborate to improve it.

Amazon isn't the form to open source its machine learning software -- Google released Tensorflow late last year -- but the company believes it has more to offer than its rival. The company says DSSTNE excels when it has less data to work with, scales better across multiple machines and is easier to deploy. It also claims its AI can solve recommendation problems and perform natural language understanding tasks two times faster than Google's library. In recent years, many of the world's biggest technology companies have invested heavily in machine learning. Google uses its AI to index your photos and improve the quality of its translations, while Facebook is exploring how to find deeper meaning in your News Feed.

Amazon is making a bigger leap into open-source technology with the unveiling of its machine-learning software DSSTNE. The newly released program is competing with Google's TensorFlow, which the search giant open-sourced last year. Amazon says DSSTNE (which stands for Deep Scalable Sparse Tensor Network Engine and is pronounced "Destiny") excels in situations where there isn't a lot of data to train the machine-learning system, whereas TensorFlow is geared for handling tons of data. DSSTNE is also faster than TensorFlow, with Amazon claiming up to 2.1 times the speed in low-data situations. The software comes from Amazon's need to make recommendations in its retail platform, which required the company to develop neural network programs.

Amazon has become the latest tech giant that's giving away some of its most sophisticated technology. Today the company unveiled DSSTNE (pronounced "destiny"), an open source artificial intelligence framework that the company developed to power its product recommendation system. Now any company, researcher, or curious tinkerer can use it for their own AI applications. It's the latest in series of projects recently open sourced by large tech companies all focused on a branch of AI called deep learning. Google, Facebook, and Microsoft have mainly used these systems for tasks like image and speech recognition.

Though Amazon Inc. doesn't make a habit of sharing its internally-produced software with the outside world, the large number of fellow web giants that had open-sourced their deep learning technology in recent months has apparently prompted a change of heart. And so the company quietly joined the fray yesterday with the release of a C library for developing neural networks that could make the task significantly faster than before for data scientists. Dubbed DSSTNE, the framework owes its speed in large part to the parallelization mechanism that Amazon included under the hood to handle distributed processing. Most alternatives execute deep learning models by running separate copies of the code on each GPU at their disposal and synchronizing the activity using some sort of orchestration mechanism. Others will assign each major element of the algorithm to a different chip, which is slightly more efficient but still doesn't make the most out of the available hardware.

Amazon has suddenly made a remarkable entrance into the world of open-source software for deep learning. Yesterday the ecommerce company quietly released a library called DSSTNE on GitHub under an open-source Apache license. Deep learning involves training artificial neural networks on lots of data and then getting them to make inferences about new data. Several technology companies are doing it -- heck, it even got some air time recently in the show "Silicon Valley." And there are already several other deep learning frameworks to choose from, including Google's TensorFlow.