It's been 20 years since IBM's supercomputer Deep Blue defeated world chess champion Gary Kasparov, in an historical first victory for artificial intelligence. What was once a futuristic concept in 1997, writes Daniel Surmacz (pictured below), COO, RTB House, has slowly become part of everyday reality. Here, Surmacz explains where the future of artificial intelligence lies. Scientists have since made huge steps towards creating a computing system that emulates the human brain's neurons, working together in a neural network to solve problems. Today, supercomputers are smart enough to easily beat not only chess players, but also succeed in similarly sophisticated games, like the 3000-year-old Chinese game of Go, and, most recently, poker challenges against multiple human pros.

Deep Learning Invades HPC While many algorithms are commonly referred to as "machine learning" (ML) or "artificial intelligence" (AI), deep learning with neural networks (NNs) has dominated the attention of the ML industry in recent years. Though numerous alternatives exist – including support vector machines, Bayesian classifiers, genetic algorithms, clustering techniques, and even decision trees – NNs have experienced a rapid increase in real-world effectiveness during recent years. Continued improvements in computing hardware help propel the ongoing expansion in the use of NNs by many industries. In fact, the demand for larger and more-powerful neural networks motivates many to leverage the unique scaling advantages provided by high-performance computing (HPC), including Cray's high-end clusters and supercomputers. Specifically, current scaling to small node counts is no longer sufficient for today's larger NN workloads, let alone the workloads of the future.

Deep learning is one of the most exciting artificial intelligence topics. It's a family of algorithms loosely based on a biological interpretation that have proven astonishing results in many areas: computer vision, natural language processing, speech recognition and more. Over the past five years, deep learning expanded to a broad range of industries. Many recent technological breakthroughs owe their existence to it. To name a few: Tesla autonomous cars, photo tagging systems at Facebook, virtual assistants such as Siri or Cortana, chatbots, object recognition cameras.

From selfies to satellites, cameras are an integral part of life. They increasingly watch over our homes and streets, and keep businesses secure inside and out. But many factors -- rain, smog, poor lighting, etc. -- can reduce the quality of images. And from identifying a thief to checking on your baby via a monitor, these factors can impair the decisions people make based on camera footage. NVIDIA Inception partner Brighter AI is focused on a fix.

Deep learning, an advanced machine-learning technique, uses layered (hence "deep") neural networks (neural nets) that are loosely modelled on the human brain. Machine learning itself is a subset of Artificial Intelligence (AI), and is broadly about teaching a computer how to spot patterns and use mountains of data to make connections without any programming to accomplish the specific task--a recommendation engine being a good example. Neural nets, on their part, enable image recognition, speech recognition, self-driving cars and smarthome automation devices, among other things.

Over the past year, I've spent most of my working time doing deep learning research and internships. And a lot of that year was making very big mistakes that helped me learn not just about ML, but about how to engineer these systems correctly and soundly. One of the main principles I learned during my time at Google Brain was that unit tests can make or break your algorithm and can save you weeks of debugging and training time. However, there doesn't seem to be a solid tutorial online on how to actually write unit tests for neural network code. Even places like OpenAI only found bugs by staring at every line of their code and try to think why it would cause a bug.

Dabblers with prominent artificial intelligence tools have been warned and/or reminded to check their dependencies because some have open vulnerabilities. That warning came from Qixue Xiao and Deyue Zhang (from Quihoo's 360 Security Research Lab), Kang Li (University of Georgia) and Weilin Xu (University of Virginia), who together wrote that "deep learning frameworks are complex and contain heavy dependencies on numerous open source packages" The three reached that conclusion after combing through the third-party packages used by the TensorFlow, Caffe, and Torch deep learning frameworks, and looking for any open bugs in those packages. They found quite a few and wrote that the frameworks are susceptible to denial-of-service, evasion attacks, or system compromise. Noting that this work stands as a preliminary study (The Register expects this means there's more to come), they still found a total of 15 vulnerabilities in the three frameworks. The largest number of bugs were found in the Open Source Computer Vision (opencv) code base: eleven CVEs in all, exploitable across all three attack classes.

Computer vision is fundamental for a broad set of Internet of Things (IoT) applications. Household monitoring systems use cameras to provide family members with a view of what's going on at home. Robots and drones use vision processing to map their environment and avoid obstacles in flight. Augmented reality glasses use computer vision to overlay important information on the user's view, and cars stitch images from multiple cameras mounted in the vehicle to provide drivers with a surround or "bird's eye" view which helps prevent collisions. Over the years, exponential improvements in device capabilities including computing power, memory capacity, power consumption, image sensor resolution, and optics have improved the performance and cost-effectiveness of computer vision in IoT applications.

PBT - like random search - starts by training many neural networks in parallel with random hyperparameters. But instead of the networks training independently, it uses information from the rest of the population to refine the hyperparameters and direct computational resources to models which show promise. This takes its inspiration from genetic algorithms where each member of the population, known as a worker, can exploit information from the remainder of the population. For example, a worker might copy the model parameters from a better performing worker. It can also explore new hyperparameters by changing the current values randomly.

The best way to think of these three terms is to think of concentric circles with artificial intelligence-- the concept that came first -- as the largest circle, with machine learning, which came next, in the middle circle and then deep learning in the center. Artificial intelligence, or AI, first got its start in 1956 when a group of scientists came up with the term at the Dartmouth Conferences. The researchers dreamed of a world where computers would have the same characteristics as human intelligence and think like we do. While we are not quite there yet, we do have a number of technologies that certainly do a decent job doing one specific task as well as, or better than, we can. Great examples are face recognition on Facebook, which will allow you back into your account if you are locked out, virtual personal assistants like Siri and websites that suggest items for you to buy, based on your past purchases.