Deep Learning
Microsoft announces expansion of Montreal research lab, new director
Microsoft plans to significantly expand its Montreal research lab and has hired a renowned artificial intelligence expert, Geoffrey Gordon, to be the lab's new research director. The company said Wednesday that it hopes to double the size of Microsoft Research Montreal within the next two years, to as many as 75 technical experts. The expansion comes as Montreal is becoming a worldwide hub for groundbreaking work in the fields of machine learning and deep learning, which are core to AI advances. "Montreal is really one of the most exciting places in AI right now," said Jennifer Chayes, a technical fellow and managing director of Microsoft Research New England, New York City and Montreal. In a meeting at the World Economic Forum in Davos, Canadian Prime Minister Justin Trudeau and Microsoft CEO Satya Nadella discussed Microsoft's ongoing investment in Canada and the expansion of the Montreal lab, including Gordon's hiring.
Deep learning improves prediction of CRISPRโCpf1 guide RNA activity
These authors contributed equally to this work. H.K.K., M.S., and S.J. performed experiments to build data sets of AsCpf1 indel frequencies. S.M. and S.Y. developed the framework, and carried out the model training and computational validation. Y.K. and S.L. made substantial contributions to the performance of the experiments including cell culture and deep-sequencing. H.H.K. conceived and designed the study.
Make It Happen
This is the first part of "An Outsider's Tour of Reinforcement Learning." If you read hacker news, you'd think that deep reinforcement learning can be used to solve any problem. Deep RL has claimed to achieve superhuman performance on Go, beat atari games, control complex robotic systems, automatically tune deep learning systems, manage queueing in network stacks, and improve energy efficiency in data centers. I personally get suspicious when audacious claims like this are thrown about in press releases, and I get even more suspicious when other researchers call into question their reproducibility. I want to take a few posts to unpack what is legitimately interesting and promising in RL and what is probably just hype.
A Robot Took My Job โ Was It a Robot or AI?
Summary: The argument in the popular press about robots taking our jobs fails in the most fundamental way to differentiate between robots and AI. Here we try to identify how each contributes to job loss and what the future of AI Enhanced Robots means for employment. There's been a lot of contradictory opinion in the press recently about future job loss from robotics and AI. They range from Bill Gates' hand wringing assertion that we should slow this down by taxing robots to Treasury Secretary Steve Mnuchin's seemingly luddite observation "In terms of artificial intelligence taking over the jobs, I think we're so far away from that that it's not even on my radar screen. I think it's 50 or 100 more years." There is essentially no effort in the popular press to differentiate'robot' from'AI'.
Hierarchical Adversarially Learned Inference
Belghazi, Mohamed Ishmael, Rajeswar, Sai, Mastropietro, Olivier, Rostamzadeh, Negar, Mitrovic, Jovana, Courville, Aaron
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model. Both the generative and inference model are trained using the adversarial learning paradigm. We demonstrate that the hierarchical structure supports the learning of progressively more abstract representations as well as providing semantically meaningful reconstructions with different levels of fidelity. Furthermore, we show that minimizing the Jensen-Shanon divergence between the generative and inference network is enough to minimize the reconstruction error. The resulting semantically meaningful hierarchical latent structure discovery is exemplified on the CelebA dataset. There, we show that the features learned by our model in an unsupervised way outperform the best handcrafted features. Furthermore, the extracted features remain competitive when compared to several recent deep supervised approaches on an attribute prediction task on CelebA. Finally, we leverage the model's inference network to achieve state-of-the-art performance on a semi-supervised variant of the MNIST digit classification task.
Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features
Madiraju, Naveen Sai, Sadat, Seid M., Fisher, Dimitry, Karimabadi, Homa
Unsupervised learning of time series data, also known as temporal clustering, is a challenging problem in machine learning. Here we propose a novel algorithm, Deep Temporal Clustering (DTC), to naturally integrate dimensionality reduction and temporal clustering into a single end-to-end learning framework, fully unsupervised. The algorithm utilizes an autoencoder for temporal dimensionality reduction and a novel temporal clustering layer for cluster assignment. Then it jointly optimizes the clustering objective and the dimensionality reduction objec tive. Based on requirement and application, the temporal clustering layer can be customized with any temporal similarity metric. Several similarity metrics and state-of-the-art algorithms are considered and compared. To gain insight into temporal features that the network has learned for its clustering, we apply a visualization method that generates a region of interest heatmap for the time series. The viability of the algorithm is demonstrated using time series data from diverse domains, ranging from earthquakes to spacecraft sensor data. In each case, we show that the proposed algorithm outperforms traditional methods. The superior performance is attributed to the fully integrated temporal dimensionality reduction and clustering criterion.
Multi-task Learning for Continuous Control
Arora, Himani, Kumar, Rajath, Krone, Jason, Li, Chong
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In addition, most reinforcement learning research on multi-task learning has been focused on discrete action spaces, which are not used for robotic control in the real-world. In this work, we apply multi-task learning methods to continuous action spaces and benchmark their performance on a series of simulated continuous control tasks. Most notably, we show that multi-task learning outperforms our baselines and alternative knowledge sharing methods.
Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning
Chen, Minghai, Wang, Sen, Liang, Paul Pu, Baltruลกaitis, Tadas, Zadeh, Amir, Morency, Louis-Philippe
With the increasing popularity of video sharing websites such as YouTube and Facebook, multimodal sentiment analysis has received increasing attention from the scientific community. Contrary to previous works in multimodal sentiment analysis which focus on holistic information in speech segments such as bag of words representations and average facial expression intensity, we develop a novel deep architecture for multimodal sentiment analysis that performs modality fusion at the word level. In this paper, we propose the Gated Multimodal Embedding LSTM with Temporal Attention (GME-LSTM(A)) model that is composed of 2 modules. The Gated Multimodal Embedding alleviates the difficulties of fusion when there are noisy modalities. The LSTM with Temporal Attention performs word level fusion at a finer fusion resolution between input modalities and attends to the most important time steps. As a result, the GME-LSTM(A) is able to better model the multimodal structure of speech through time and perform better sentiment comprehension. We demonstrate the effectiveness of this approach on the publicly-available Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis (CMU-MOSI) dataset by achieving state-of-the-art sentiment classification and regression results. Qualitative analysis on our model emphasizes the importance of the Temporal Attention Layer in sentiment prediction because the additional acoustic and visual modalities are noisy. We also demonstrate the effectiveness of the Gated Multimodal Embedding in selectively filtering these noisy modalities out. Our results and analysis open new areas in the study of sentiment analysis in human communication and provide new models for multimodal fusion.
Nudity Detection and Abusive Content Classifiers -- Research and Use cases
Web 2.0 revolution has led to the explosion of content generated every day on the internet. Social sharing platforms such as Facebook, Twitter, Instagram etc. have seen astonishing growth in their daily active users but have been at their split ends when it comes to monitoring the content generated by their users. Users are uploading inappropriate content such as nudity or using abusive language while commenting on posts. Such behavior leads to social issues like bullying and revenge porn and also hampers the authenticity of the platform. However, the pace at which the content is generated online today is so high that it is nearly impossible to monitor everything manually.