On May 11, 1997, a computer showed that it could outclass a human in that most human of pursuits: playing a game. The human was World Chess Champion Garry Kasparov, and the computer was IBM's Deep Blue, which had begun life at Carnegie Mellon University as a system called ChipTest. One of Deep Blue's creators, Murray Campbell, talked to the IDG News Service about the other things computers have learned to do as well as, or better than, humans, and what that means for our future. What follows is an edited version of that conversation. IDGNS: Is it true that you and Deep Blue joined IBM at the same time?

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

We introduce a method to stabilize Generative Adversarial Networks (GANs) by defining the generator objective with respect to an unrolled optimization of the discriminator. This allows training to be adjusted between using the optimal discriminator in the generator's objective, which is ideal but infeasible in practice, and using the current value of the discriminator, which is often unstable and leads to poor solutions. We show how this technique solves the common problem of mode collapse, stabilizes training of GANs with complex recurrent generators, and increases diversity and coverage of the data distribution by the generator.

We propose an algorithm to separate simultaneously speaking persons from each other, the "cocktail party problem", using a single microphone. Our approach involves a deep recurrent neural networks regression to a vector space that is descriptive of independent speakers. Such a vector space can embed empirically determined speaker characteristics and is optimized by distinguishing between speaker masks. We call this technique source-contrastive estimation. The methodology is inspired by negative sampling, which has seen success in natural language processing, where an embedding is learned by correlating and de-correlating a given input vector with output weights. Although the matrix determined by the output weights is dependent on a set of known speakers, we only use the input vectors during inference. Doing so will ensure that source separation is explicitly speaker-independent. Our approach is similar to recent deep neural network clustering and permutation-invariant training research; we use weighted spectral features and masks to augment individual speaker frequencies while filtering out other speakers. We avoid, however, the severe computational burden of other approaches with our technique. Furthermore, by training a vector space rather than combinations of different speakers or differences thereof, we avoid the so-called permutation problem during training. Our algorithm offers an intuitive, computationally efficient response to the cocktail party problem, and most importantly boasts better empirical performance than other current techniques.

KDnuggets has taken seriously its role to keep up with the newest releases of major deep learning projects, and in the recent past we have seen landmark such releases from major technology giants and as well as universities and research labs. While Microsoft, Yahoo!, AMPLabs, and others have all contributed outstanding projects in their own right, the landscape was most impacted in November, 2015, with the release of what is now the most popular open source machine learning library on Github by a wide margin, Google's TensorFlow. I wrote in the early days after its release of my initial dissatisfaction with the project, based primarily on the lack of distributed training capabilities (especially given that such capabilities were directly alluded to in the accompanying whitepaper's title). There were also a few other - lesser - "issues" I had with it, but the central point of contention was that it was single node only. This original post was polarizing, with many people upset at my "dismissal" of the tech powerhouse's latest offering (a closer read would reveal that I did not, in any way, dismiss it).

Starting this week, I'll be doing a new series called Deep Learning Research Review. Every couple weeks or so, I'll be summarizing and explaining research papers in specific subfields of deep learning. This week I'll begin with Generative Adversarial Networks. According to Yann LeCun, "adversarial training is the coolest thing since sliced bread". I'm inclined to believe so because I don't think sliced bread ever created this much buzz and excitement within the deep learning community.

On May 11, 1997, a computer showed that it could outclass a human in that most human of pursuits: playing a game. The human was World Chess Champion Garry Kasparov, and the computer was IBM's Deep Blue, which had begun life at Carnegie Mellon University as a system called ChipTest. One of Deep Blue's creators, Murray Campbell, talked to the IDG News Service about the other things computers have learned to do as well as, or better than, humans, and what that means for our future. What follows is an edited version of that conversation. IDGNS: Is it true that you and Deep Blue joined IBM at the same time?

Dr. Yoshua Bengio, one of the key researchers credited with establishing the field of deep learning, announced today that he will serve as a strategy adviser for legal tech company Botler AI, a Montreal-based startup that runs a free chatbot to assist users as they navigate the legal procedures required for immigration. "I knew this was an idea that we needed to take to the next level," Bengio said in today's press release. "I want AI to be developed towards positive social impact and this is one place where beneficial AI must happen." The techniques pioneered by Bengio and others have led to breakthroughs in natural language comprehension and translation, abilities that are critical to this kind of chatbot. "Deep Learning could help provide affordable or even free legal services to people who might otherwise not be able to have access," he said.

Artificial Intelligence (AI) pioneer Nvidia has announced it will train 100,000 developers in "deep learning" to bolster health care research and improve treatment in diseases like cancer. Deep learning is Nvidia's term for machine learning, the idea of pushing computers to learn the way a human would in order to progress what many are calling the next revolution in technology – machines that "think" like humans. Over the past decade, it's given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. In the past, cancer research institutes have looked into using Nvidia's latest advances in AI and deep learning to help pathologists with their overwhelming tasks. One project, Led by Andrew Beck, associate professor of pathology and director of bioinformatics at BIDMC, used an Nvidia Tesla K80 GPU supercomputer chip to speed up the process of training their computational models in breast cancer diagnosis.

While feats of Deep Learning has been gathering much attention, there were also breakthroughs in a related technology of Recurrent Neural Networks (RNN). RNNs hold great promise for learning general sequences, and have applications for text analysis, handwriting recognition and even machine translation. RNN is learning to paint house numbers (Andrej Karpathy) See a fantastic post by Andrej Karpathy, "The Unreasonable Effectiveness of Recurrent Neural Networks" where he uses RNNs to do amazing stuff like paint house numbers in this image, or generate text in the style of Paul Graham, Shakespeare, and even Latex. See below an excellent tutorial "General Sequence Learning using Recurrent Neural Networks" by Alec Radford, Indico Head of Research, who led a workshop on general sequence learning using recurrent neural networks at Next.ML in San Francisco, Feb 2015. Alec introduces RNNs and sketches how to implement them and cover the tricks necessary to make them work well.