Watson started as a follow-on project to IBM DeepBlue, the computer and AI program that defeated world chess champion Gary Kasparov. DeepBlue demonstrated that a computer could defeat a human in chess, a game with well-defined rules and limited, fully visible solutions. The real world, however, is much more complicated: information often is unstructured, problems ill defined, and solutions probabilistic at best. To equip AI to deal with the real world, IBM challenged its computer and data scientists to create a program that could defeat human contestants at Jeopardy!, a quiz show requiring answers to natural language questions over broad domains of knowledge otherwise known as unstructured data. As a quick refresher, artificial intelligence can be divided into three categories, as shown above.1The
We propose an approach to address two issues that commonly occur during training of unsupervised GANs. First, since GANs use only a continuous latent distribution to embed multiple classes or clusters of data, they often do not correctly handle the structural discontinuity between disparate classes in a latent space. Second, discriminators of GANs easily forget about past generated samples by generators, incurring instability during adversarial training. We argue that these two infamous problems of unsupervised GAN training can be largely alleviated by a learnable memory network to which both generators and discriminators can access. Generators can effectively learn representation of training samples to understand underlying cluster distributions of data, which ease the structure discontinuity problem. At the same time, discriminators can better memorize clusters of previously generated samples, which mitigate the forgetting problem. We propose a novel end-to-end GAN model named memoryGAN, which involves a memory network that is unsupervisedly trainable and integrable to many existing GAN models. With evaluations on multiple datasets such as Fashion-MNIST, CelebA, CIFAR10, and Chairs, we show that our model is probabilistically interpretable, and generates realistic image samples of high visual fidelity. The memoryGAN also achieves the state-of-the-art inception scores over unsupervised GAN models on the CIFAR10 dataset, without any optimization tricks and weaker divergences.
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.
Machine learning is everywhere these days. It's in your email account filtering out spam and other emails you don't want to read. It's in your connected car helping the voice-controlled interface understand you. Right now, Amazon, Google, IBM, and Microsoft are the biggest players battling to dominate the very fast-growing machine learning cloud services market. IBM further strengthened its position in the market with the recent acquisition of AlchemyAPI, a leading deep learning-based machine learning services platform.
IBM will launch a Korean version of its AI platform Watson next year in cooperation with local IT service vendor SK C&C, the companies have announced. SK announced Monday that it signed a cooperation agreement with Big Blue on May 4 and will together build an integrated system to market Watson in South Korea. They will develop Korean data analysis solutions based on machine learning and natural language semantic analysis technology for Watson within this year, and will commercialise it sometime in the first half of 2017, SK said. IBM and SK will also build a "Watson Cloud Platform" at the Korean company's datacentre in Pangyo -- the local version of Silicon Valley -- that IT developers and managers can access to make their own applications. For example, an open market business can apply the Watson solution to its product search features to make a personalized contents recommendation solution.