The American elections have been a hot topic in the office as we contemplate expanding our presence to the US. Since its debut in March, we have been entertained by the senseless tweets of DeepDrumpf, a Twitter bot created by Bradley Hayes, a postdoc at MIT. DeepDrumpf was trained on a few hours worth of transcripts of victory speeches and debates from the president elect using deep learning techniques. The tweets were constructed character by character and inspired by recurrent neural network models that had been previously employed to mimic Shakespearean speech. Although not the most sophisticated use of deep learning that we've seen, we must hand it to him for originality and capturing the zeitgeist.

Amazon Lex, Amazon Polly, and Amazon Rekognition are based on the same proven, highly scalable Amazon technology built by the thousands of deep learning and machine learning experts across the company. Amazon AI services all provide high-quality, high-accuracy AI capabilities that are scalable and cost-effective. Amazon AI services are fully managed services so there are no deep learning algorithms to build, no machine learning models to train, and no up-front commitments or infrastructure investments required. This frees developers to focus on defining and building an entirely new generation of apps that can see, hear, speak, understand, and interact with the world around them. To learn more about Amazon Lex, Amazon Polly, or Amazon Rekognition, visit: https://aws.amazon.com/amazon-ai

The Fourth Industrial Revolution: what it means... We stand on the brink of a technological revolution that will fundamentally alter the way we live, work, and relate to one another. In its scale, scope, and complexity, the transformation will be unlike anything humankind has experienced before. We do not yet know just how it will unfold, but one... We stand on the brink of a technological revolution that will fundamentally alter the way we live, work, and relate to one another.

The first wave of bots were based on rules programmed into the software and were used to automate simple, repetitive tasks. However, bots have advanced to the point where they are streamlining support cases, explaining frequently asked questions, scheduling appointments, and completing orders. Originally, these tasks required multiple inputs from a human to answer rule-based logic questions. But with advancements in AI and leveraging deep learning, bots are now able to perform more complex tasks and write their own commands on the fly using massive data sets to answer even more complex queries.

Amazon Web Services is bringing its consumer technology artificial intelligence capabilities to the enterprise, unveiling three new AI services at its re:Invent developer conference on Wednesday. The move is intended to make it easier for developers to work with AI, which normally requires vast amounts of data and a specialization in machine learning and neural networks, according to the company. Now, AWS is making AI more accessible to application developers, offering its deep learning algorithms and technology as managed services. The company unveiled Amazon Lex, which uses the technology that powers its smart device Alexa, to let developers build "conversational user experiences" for web, mobile and connected devices; Amazon Polly, which turns text into speech in 47 different voices for 24 languages; and Amazon Rekognition, which uses deep learning and facial recognition to add image analysis to applications. Amazon Web Services is bringing its consumer technology artificial intelligence capabilities to the enterprise, unveiling three new AI services at its re:Invent developer conference on Wednesday.

In this work we investigate intra-day patterns of activity on a population of 7,261 users of mobile health wearable devices and apps. We show that: (1) using intra-day step and sleep data recorded from passive trackers significantly improves classification performance on self-reported chronic conditions related to mental health and nervous system disorders, (2) Convolutional Neural Networks achieve top classification performance vs. baseline models when trained directly on multivariate time series of activity data, and (3) jointly predicting all condition classes via multi-task learning can be leveraged to extract features that generalize across data sets and achieve the highest classification performance.

Until recently, research on artificial neural networks was largely restricted to systems with only two types of variable: Neural activities that represent the current or recent input and weights that learn to capture regularities among inputs, outputs and payoffs. There is no good reason for this restriction. Synapses have dynamics at many different time-scales and this suggests that artificial neural networks might benefit from variables that change slower than activities but much faster than the standard weights. These "fast weights" can be used to store temporary memories of the recent past and they provide a neurally plausible way of implementing the type of attention to the past that has recently proved very helpful in sequence-to-sequence models. By using fast weights we can avoid the need to store copies of neural activity patterns.

Current deep learning architectures are growing larger in order to learn from complex datasets. These architectures require giant matrix multiplication operations to train millions of parameters. Conversely, there is another growing trend to bring deep learning to low-power, embedded devices. The matrix operations, associated with both training and testing of deep networks, are very expensive from a computational and energy standpoint. We present a novel hashing based technique to drastically reduce the amount of computation needed to train and test deep networks. Our approach combines recent ideas from adaptive dropouts and randomized hashing for maximum inner product search to select the nodes with the highest activation efficiently. Our new algorithm for deep learning reduces the overall computational cost of forward and back-propagation by operating on significantly fewer (sparse) nodes. As a consequence, our algorithm uses only 5% of the total multiplications, while keeping on average within 1% of the accuracy of the original model. A unique property of the proposed hashing based back-propagation is that the updates are always sparse. Due to the sparse gradient updates, our algorithm is ideally suited for asynchronous and parallel training leading to near linear speedup with increasing number of cores. We demonstrate the scalability and sustainability (energy efficiency) of our proposed algorithm via rigorous experimental evaluations on several real datasets.

Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of interest for shallow learning, e.g., logistic regression. However, the independent sampling for dropout could be suboptimal for the sake of convergence. In this paper, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named \textbf{evolutional dropout}) that computes the sampling probabilities on-the-fly from a mini-batch of examples. Empirical studies on several benchmark datasets demonstrate that the proposed dropouts achieve not only much faster convergence and but also a smaller testing error than the standard dropout. For example, on the CIFAR-100 data, the evolutional dropout achieves relative improvements over 10\% on the prediction performance and over 50\% on the convergence speed compared to the standard dropout.

If you've been following my articles up to now, you'll begin to perceive, what's apparent to many advanced practitioners of Deep Learning (DL), is the emergence of Game Theoretic concepts in the design of newer architectures. This makes intuitive sense for two reasons. The first intuition is that DL systems will eventually need to tackle situations with imperfect knowledge. In fact we've already seen this in DeepMind's AlphaGo that uses partial knowledge to tactically and strategically best the world-best human in the game of Go. The second intuition is that systems will not remain monolithic as they are now, but rather would involve multiple coordinating (or competing) cliques of DL systems.