This year was a big one for artificial intelligence, machine learning, deep learning and all the related technologies. Thanks to innovations and breakthroughs, the industry took great leaps both for the better and the worse this year. Though we're still a long way from Singularity or Skynet (or Genisys or whatever else you want to call a robot and artificial intelligence invasion), we can all acknowledge that the lines between man and machine became a little bit more blurred in 2016. Here are some of the hottest things AI had in stock for us in the past year. When Google DeepMind's AlphaGo beat world champion Lee Sedol at Go four out of five times, it wasn't just another chapter in the endless saga of board game matchups between man and machine--it was a watershed moment.

The race is on to attract as much expertise in artificial intelligence as possible at tech companies large and small, and more than a few Silicon Valley giants are looking inward to convert tech talent they already possess into the AI resources they increasingly need. Facebook has its own AI course, which is oversubscribed, according to a new report by Wired, and which is led by one of the leading AI researchers in the world. Facebook's Larry Zitnick, who is a key leader at the social networking company's Artificial Intelligence Research Lab, as well as a Microsoft Research and CMU Robotics alum, teaches a class on deep learning for Facebook employees that draws over-capacity crowds. Zitnick's course sparks strong competition among engineers who already rank among the best in the world, each vying to come to grips with and excel at a field outside of their original purview, but one that few fail to recognize is the hottest in tech. On the other hand, AI and deep learning increasingly touch all aspects of the technology business, so experts with understanding of where the overlap might prove most useful in their own original discipline are also going to be very much in demand. There are external efforts underway to help create more of these polyglot deep learning pros, including at online educational firms like Udacity, but new talent isn't rolling in fast enough from outside sources, traditional and non-traditional alike.

THE world reeled when Lee Sedol – one of the great modern players of the ancient board game Go – was beaten by Google's DeepMind artificial intelligence (AI) program, AlphaGo. The AI managed to outmaneuver Lee at his own game, one which rewards players' strategic judgment and creative analyses. To achieve this, DeepMind provided AlphaGo with the basic framework of the game, recordings of previous games and made it play itself continuously. The software mimics the processes of human learning – and as it went along, AlphaGo learned to be a better player over time. The day of the face-off, AlphaGo beat Lee four games to one and was awarded the highest Go game-master ranking.

Chia-Chiunn Ho was eating lunch inside Facebook headquarters, at the Full Circle Cafe, when he saw the notice on his phone: Larry Zitnick, one of the leading figures at the Facebook Artificial Intelligence Research lab, was teaching another class on deep learning. Ho is a 34-year-old Facebook digital graphics engineer known to everyone as "Solti," after his favorite conductor. He couldn't see a way of signing up for the class right there in the app. So he stood up from his half-eaten lunch and sprinted across MPK 20, the Facebook building that's longer than a football field but feels like a single room. "My desk is all the way at the other end," he says. Sliding into his desk chair, he opened his laptop and surfed back to the page.

These are some the most elite academic journals in the world. And last year, one tech company, Alphabet's Google, published papers in all of them. The unprecedented run of scientific results by the Mountain View search giant touched on everything from ophthalmology to computer games to neuroscience and climate models. For Google, 2016 was an annus mirabilis during which its researchers cracked the top journals and set records for sheer volume. Behind the surge is Google's growing investment in artificial intelligence, particularly "deep learning," a technique whose ability to make sense of images and other data is enhancing services like search and translation (see "10 Breakthrough Technologies 2013: Deep Learning").

Neural network created in SAS Visual Data Mining and Machine Learning 8.1 Artificial intelligence, machine learning and neural networks-based deep learning are concepts that have recently come to dominate venture capital funding, startup formation, promotion and exits and policy discussions. The highly-publicized triumphs over humans in Go and Poker, rapid progress in speech recognition, image identification, and language translation, and the proliferation of talking and texting virtual assistants and chatbots, have helped inflate the market cap of Apple (#1 as of February 17), Google (#2), Microsoft (#3), Amazon (#5), and Facebook (#6). While these companies dominate the headlines--and the war for the relevant talent--other companies that have been analyzing data or providing tools for analysis for years are also capitalizing on recent AI advances. A case in point are Equifax and SAS: The former developing deep learning tools to improve credit scoring and the latter adding new deep learning functionality to its data mining tools and offering a deep learning API. Both companies have a lot of experience in what they do.

These are some the most elite academic journals in the world. And last year, one tech company, Alphabet's Google, published papers in all of them. The unprecedented run of scientific results by the Mountain View search giant touched on everything from ophthalmology to computer games to neuroscience and climate models. For Google, 2016 was an annus mirabilis during which its researchers cracked the top journals and set records for sheer volume. Behind the surge is Google's growing investment in artificial intelligence, particularly "deep learning," a technique whose ability to make sense of images and other data is enhancing services like search and translation (see "10 Breakthrough Technologies 2013: Deep Learning").

It was just a friendly little argument about the fate of humanity. Demis Hassabis, a leading creator of advanced artificial intelligence, was chatting with Elon Musk, a leading doomsayer, about the perils of artificial intelligence. They are two of the most consequential and intriguing men in Silicon Valley who don't live there. Hassabis, a co-founder of the mysterious London laboratory DeepMind, had come to Musk's SpaceX rocket factory, outside Los Angeles, a few years ago. They were in the canteen, talking, as a massive rocket part traversed overhead. Musk explained that his ultimate goal at SpaceX was the most important project in the world: interplanetary colonization. Hassabis replied that, in fact, he was working on the most important project in the world: developing artificial super-intelligence. Musk countered that this was one reason we needed to colonize Mars--so that we'll have a bolt-hole if A.I. goes rogue and turns on humanity. Amused, Hassabis said that A.I. would simply follow humans to Mars. This did nothing to soothe Musk's anxieties (even though he says there are scenarios where A.I. wouldn't follow). An unassuming but competitive 40-year-old, Hassabis is regarded as the Merlin who will likely help conjure our A.I. children. The field of A.I. is rapidly developing but still far from the powerful, self-evolving software that haunts Musk. Facebook uses A.I. for targeted advertising, photo tagging, and curated news feeds. Microsoft and Apple use A.I. to power their digital assistants, Cortana and Siri. Google's search engine from the beginning has been dependent on A.I. All of these small advances are part of the chase to eventually create flexible, self-teaching A.I. that will mirror human learning. Some in Silicon Valley were intrigued to learn that Hassabis, a skilled chess player and former video-game designer, once came up with a game called Evil Genius, featuring a malevolent scientist who creates a doomsday device to achieve world domination.

Inspired by biophysical principles underlying nonlinear dendritic computation in neural circuits, we develop a scheme to train deep neural networks to make them robust to adversarial attacks. Our scheme generates highly nonlinear, saturated neural networks that achieve state of the art performance on gradient based adversarial examples on MNIST, despite never being exposed to adversarially chosen examples during training. Moreover, these networks exhibit unprecedented robustness to targeted, iterative schemes for generating adversarial examples, including second-order methods. We further identify principles governing how these networks achieve their robustness, drawing on methods from information geometry. We find these networks progressively create highly flat and compressed internal representations that are sensitive to very few input dimensions, while still solving the task. Moreover, they employ highly kurtotic weight distributions, also found in the brain, and we demonstrate how such kurtosis can protect even linear classifiers from adversarial attack.

Recently, deep learning (DL) methods have been introduced very successfully into human activity recognition (HAR) scenarios in ubiquitous and wearable computing. Especially the prospect of overcoming the need for manual feature design combined with superior classification capabilities render deep neural networks very attractive for real-life HAR application. Even though DL-based approaches now outperform the state-of-the-art in a number of recognitions tasks of the field, yet substantial challenges remain. Most prominently, issues with real-life datasets, typically including imbalanced datasets and problematic data quality, still limit the effectiveness of activity recognition using wearables. In this paper we tackle such challenges through Ensembles of deep Long Short Term Memory (LSTM) networks. We have developed modified training procedures for LSTM networks and combine sets of diverse LSTM learners into classifier collectives. We demonstrate, both formally and empirically, that Ensembles of deep LSTM learners outperform the individual LSTM networks. Through an extensive experimental evaluation on three standard benchmarks (Opportunity, PAMAP2, Skoda) we demonstrate the excellent recognition capabilities of our approach and its potential for real-life applications of human activity recognition.