Lecture 1 introduces the concept of Natural Language Processing (NLP) and the problems NLP faces today. The concept of representing words as numeric vectors is then introduced, and popular approaches to designing word vectors are discussed. This lecture series provides a thorough introduction to the cutting-edge research in deep learning applied to NLP, an approach that has recently obtained very high performance across many different NLP tasks including question answering and machine translation. It emphasizes how to implement, train, debug, visualize, and design neural network models, covering the main technologies of word vectors, feed-forward models, recurrent neural networks, recursive neural networks, convolutional neural networks, and recent models involving a memory component. For additional learning opportunities please visit: http://stanfordonline.stanford.edu/

In September 2010, a three-person AI startup called DeepMind Technologies launched in London, with the goal of "solving intelligence." Four years later, Google acquired the company for $500 million. And by 2016, it had achieved a major victory in AI: Mastering the complex game of Go. This story represents the fantasy of many AI researchers, eager to launch their own ventures in the AI startup space. But the field has become saturated, and the terms "AI," "deep learning," and "machine learning" are often overhyped and misunderstood.

Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the over-confidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.

Over 50 million scholarly articles have been published: they constitute a unique repository of knowledge. In particular, one may infer from them relations between scientific concepts, such as synonyms and hyponyms. Artificial neural networks have been recently explored for relation extraction. In this work, we continue this line of work and present a system based on a convolutional neural network to extract relations. Our model ranked first in the SemEval-2017 task 10 (ScienceIE) for relation extraction in scientific articles (subtask C).

Being able to predict whether a song can be a hit has impor- tant applications in the music industry. Although it is true that the popularity of a song can be greatly affected by exter- nal factors such as social and commercial influences, to which degree audio features computed from musical signals (whom we regard as internal factors) can predict song popularity is an interesting research question on its own. Motivated by the recent success of deep learning techniques, we attempt to ex- tend previous work on hit song prediction by jointly learning the audio features and prediction models using deep learning. Specifically, we experiment with a convolutional neural net- work model that takes the primitive mel-spectrogram as the input for feature learning, a more advanced JYnet model that uses an external song dataset for supervised pre-training and auto-tagging, and the combination of these two models. We also consider the inception model to characterize audio infor- mation in different scales. Our experiments suggest that deep structures are indeed more accurate than shallow structures in predicting the popularity of either Chinese or Western Pop songs in Taiwan. We also use the tags predicted by JYnet to gain insights into the result of different models.

DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints between different modalities, and cannot take advantage of unlabeled data. This paper proposes Cross-modal Deep Metric Learning with Multi-task Regularization (CDMLMR), which integrates quadruplet ranking loss and semi-supervised contrastive loss for modeling cross-modal semantic similarity in a unified multi-task learning architecture. The quadruplet ranking loss can model the semantically similar and dissimilar constraints to preserve cross-modal relative similarity ranking information. The semi-supervised contrastive loss is able to maximize the semantic similarity on both labeled and unlabeled data. Compared to the existing methods, CDMLMR exploits not only the similarity ranking information but also unlabeled cross-modal data, and thus boosts cross-modal retrieval accuracy.

The science of deep learning, a sub-discipline of artificial intelligence(AI), is only a recent development in the grand scheme of things, but during its short existence, it has been producing some impressive technological achievements. Advances in image recognition, language understanding, and translation have led to the development of virtual assistants, smart home speakers, gains in cybersecurity, and are leading the charge toward autonomous driving. Now, companies have found a way to use those AI smarts to fight cancer. Deep learning involves the construction of artificial neural networks, using software and complex algorithms to recreate the capacity of the human brain to learn. These learning computers have a particular knack for sifting through vast amounts of data and recognizing patterns, getting smarter as they go.

This post is by Chris Basoglu, Partner Engineering Manager in the AI & Research group at Microsoft. We are delighted to announce that Microsoft has brought Microsoft Cognitive Toolkit version 2.0 out of beta and is making the first release candidate available today. The toolkit, previously known as CNTK, is a system for deep learning used to speed advances in areas such as speech and image recognition and search relevance on CPUs and NVIDIA GPUs. Cognitive Toolkit can be used on-premises or in the cloud with Azure GPUs. Cognitive Toolkit is being used extensively by a wide variety of Microsoft products, by companies worldwide with a need to deploy deep learning at scale, and by students interested in the very latest algorithms and techniques.

For those of you who live in China, recent news about the departure of the Chief artificial intelligence (AI) scientist at Baidu, Andrew Ng, was rather a surprise (for our lovely American readers, Baidu is often dubbed the Google equivalent in China). Mr. Ng was the founder of Google Brain and a leading expert in deep learning, so news that he had jumped ship was alarming because in the world of AI investing, China is up there right behind the U.S. according to a recent article in the South China Morning Post. While China's investment in AI pales in comparison to the U.S., it's still so meaningful that "experts are warning" AI is on the verge of becoming a bubble in China which corresponds with our recent article on "investing in AI stocks". With China's AI market expected to reach $9 billion by 2020, it seems like we're hearing some conflicting messages from over there. Fortunately for our readers, we have some foreign correspondents living in those parts so we thought we would go through the China Money Network list of the top-10 AI companies in China to see what's happening.

Check out Fundamentals of Deep Learning by Nikhil Buduma to learn about key concepts in this complex and exciting field. Deep learning is enjoying unprecedented success in a wide variety of commercial applications. Around 10 years ago, very few practitioners could have predicted that deep learning-powered systems would surpass human-level performance in computer vision and speech recognition tasks. At Lawrence Berkeley National Laboratory, we are confronted with some of the most challenging data analytics problems in science. While there are similarities between commercial and scientific applications in terms of the overall analytics tasks (classification, clustering, anomaly detection, etc.), a priori, there is no reason to believe that the underlying complexity of scientific data sets would be comparable to ImageNet. Are deep learning methods powerful enough to produce state-of-the-art performance for scientific analytics tasks?