With the emergence of Deep Learning as the dominant paradigm for Artificial Intelligence based systems, one open question that seems to be neglected is "What guidelines do we have in architecting software that uses Deep Learning?" If all the innovative companies like Google are on a exponential adoption curve to incorporate Deep Learning in every thing they do, then what perhaps is the software architecture that holds this all together? The folks at Google wrote a paper (a long time ago, meaning 2014), "Machine Learning: The High-Interest Credit Card of Technical Debt" that enumerates many of the difficulties that we need to consider when building software that consists of machine learning or deep learning sub-components. Contrary to popular perception that that Deep Learning systems can be "self-driving". There is a massive ongoing maintenance cost when machine learning is used.

This is the third installment of a new series called Deep Learning Research Review. Every couple weeks or so, I'll be summarizing and explaining research papers in specific subfields of Deep Learning. This week focuses on applying Deep Learning to Natural Language Processing. The last post was about reinforcement learning and the post before was on generative adversarial networks. Natural Language Processing (NLP) is all about creating systems that process or "understand" language in order to perform certain tasks. The traditional approach to NLP involved a lot of domain knowledge of linguistics itself. Understanding terms such as phonemes and morphemes was pretty standard, as there are whole linguistic classes dedicated to their study.

People are constantly trying to find more'human' ways to interact with technology. Could the same be achieved for how we approach our health too? After the recent Deep Learning in Healthcare conference held in London, it is clear that technology has the capacity to transform healthcare as we know it. Deep learning (also known as machine learning or artificial intelligence) refers to a set of algorithms whereby a computational system is "trained" to process data by being tested on some information, which it then "learns" from. This results in a computer which has the ability to assimilate, process and present data to humans, eliminating work for us humans in the process.

Meaningful engagement on Twitter is something that many of us aspire, yet it is a difficult task to achieve. A new tool, called Post Intelligence will use artificial intelligence (AI) to help you tweet better and learn what your followers like or don't. The company's AI bot -- Pi, crawls the Twitter feeds of both your followers and the people you follow to find out relevant content that your followers may like, to help you create better posts. "Pi helps you make great posts and gain followers by creating a deep learning model just for you. Pi learns what you post about and what works well with your audience. Once it understands you and your posting patterns, Pi will suggest great content, optimal times and even personalized topics, so you will always have something to say," the company's website says.

As rival chip makers work to target workloads using artificial intelligence, Intel Corp. is upping the ante by unifying its own efforts under a dedicated business group announced today. The division will be led by Naveen Rao, a veteran of the semiconductor industry with a Ph.D. in computational neuroscience. He came aboard last year after Intel acquired his deep learning startup, Nervana Systems Inc., in a deal reportedly worth over $400 million. The executive wrote in a blog post that the Artificial Intelligence Products Group will work to "align resources from across the company to include engineering, labs, software and more" around a common roadmap. Intel, which has acquired Altera Corp., Nervana, Movidius Ltd. and other intellectual property in recent years, late last year accelerated its efforts in AI, announcing new chips, software and partnerships.

It is now table stakes for artificial intelligence algorithms to "learn" about the world around them. The next level: For AI bots to learn how to talk to each other -- and develop their own shared language. New research released last week by OpenAI, the artificial intelligence nonprofit lab founded by Elon Musk and Y Combinator president Sam Altman, details how they're training AI bots to create their own language, based on trial and error, as the bots move around a set environment. This is different from how artificial intelligence algorithms typically learn -- using large sets of data, like to recognize a dog by taking in thousands of pictures of dogs. The world the researchers created for the AI bots to learn in is a computer simulation of a simple, two-dimensional white square.

A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional method which we call Smart Augmentation and we show how to use it to increase the accuracy and reduce overfitting on a target network. Smart Augmentation works by creating a network that learns how to generate augmented data during the training process of a target network in a way that reduces that networks loss. This allows us to learn augmentations that minimize the error of that network. Smart Augmentation has shown the potential to increase accuracy by demonstrably significant measures on all datasets tested. In addition, it has shown potential to achieve similar or improved performance levels with significantly smaller network sizes in a number of tested cases.

Natural language processing (NLP) is all about creating systems that process or "understand" language in order to perform certain tasks. The traditional approach to NLP involved a lot of domain knowledge of linguistics itself. Understanding terms such as phonemes and morphemes were pretty standard as there are whole linguistic classes dedicated to their study. Let's look at how traditional NLP would try to understand the following word. Let's say our goal is to gather some information about this word (characterize its sentiment, find its definition, etc).

Debates between humans and computers start with mechanical turk. The mechanism allows to hide a chess player inside the machine. Thus, the turk operates while hiding master playing chess. So, there is no intelligence for this ancient example. Still, this fake machine shows expectations of 18th century people for an intelligent system to involve in daily life.

Pokémon has been an enormous success around the globe for more than 20 years. In this paper, I tackle the "Who's that Pokémon?" challenge from a machine learning perspective. I propose a machine learning pre-processing and classification pipeline, using convolutional neural networks for classification of Pokémon sprites. Since they were invented[1], computers became increasingly present in our everyday life. Initially restricted to mathematical problem-solving and military applications in ballistics and cryptography, their applications become more diverse by the minute. As of today, machines beat humans in lots of tasks, one of the most recent being AlphaGo's victory over the Go world champion (Go Game Guru, 2017). This achievement is a testament to the remarkable advances sustained by machines towards intelligent applications. Go, with its almost infinite combinations[2], is not an easy problem to solve by "brute force"[3], the strategy usually employed by computers against humans in other perfect information games. But do not despair, for not all is lost in our fight against our future robot overlords, as computers still struggle with a task that humans were quite literally born to do: image and pattern recognition.