This article is also available in Japanese and Simplified Chinese. Lionbridge AI has assembled a wealth of resources for machine learning and natural language processing activities. In our previous articles, we explained why datasets are such an integral part of machine learning and natural language processing. Without training datasets, machine-learning algorithms would have no way of learning how to do text mining, text classification, or categorize products. This article is the ultimate list of open datasets for machine learning.

In this issue of Communications, as evidenced by the cover and lead article, we celebrate the latest recipients of the ACM A.M. Turing Award. Yoshua Bengio, Yann LeCun, and Geoffrey Hinton carried out pioneering work in deep learning that has touched all our lives. As Turing Laureates, they now join the eminent group of technology visionaries recognized with the world's highest distinction in computing. The Turing Award is one of a suite of professional honors ACM bestows annually to recognize technical achievements that have made significant contributions to our field. This month, I will have the pleasure of joining the awardees, ACM Fellows, and other luminaries in San Francisco for the ACM Awards Banquet.

When Geoffrey Hinton started doing graduate student work on artificial intelligence at the University of Edinburgh in 1972, the idea that it could be achieved using neural networks that mimicked the human brain was in disrepute. Computer scientists Marvin Minsky and Seymour Papert had published a book in 1969 on Perceptrons, an early attempt at building a neural net, and it left people in the field with the impression that such devices were nonsense. "It didn't actually say that, but that's how the community interpreted the book," says Hinton who, along with Yoshua Bengio and Yann LeCun, will receive the 2018 ACM A.M. Turing award for their work that led deep neural networks to become an important component of today's computing. "People thought I was just completely crazy to be working on neural nets." Even in the 1980s, when Bengio and LeCun entered graduate school, neural nets were not seen as promising.

Reading Comprehension has received significant attention in recent years as high quality Question Answering (QA) datasets have become available. Despite state-of-the-art methods achieving strong overall accuracy, Multi-Hop (MH) reasoning remains particularly challenging. To address MH-QA specifically, we propose a Deep Reinforcement Learning based method capable of learning sequential reasoning across large collections of documents so as to pass a query-aware, fixed-size context subset to existing models for answer extraction. Our method is comprised of two stages: a linker, which decomposes the provided support documents into a graph of sentences, and an extractor, which learns where to look based on the current question and already-visited sentences. The result of the linker is a novel graph structure at the sentence level that preserves logical flow while still allowing rapid movement between documents. Importantly, we demonstrate that the sparsity of the resultant graph is invariant to context size. This translates to fewer decisions required from the Deep-RL trained extractor, allowing the system to scale effectively to large collections of documents. The importance of sequential decision making in the document traversal step is demonstrated by comparison to standard IE methods, and we additionally introduce a BM25-based IR baseline that retrieves documents relevant to the query only. We examine the integration of our method with existing models on the recently proposed QAngaroo benchmark and achieve consistent increases in accuracy across the board, as well as a 2-3x reduction in training time.

Nowadays, online learning is an appealing learning paradigm, which is of great interest in practice due to the recent emergence of large scale applications such as online advertising placement and online web ranking. Standard online learning assumes a finite number of samples while in practice data is streamed infinitely. In such a setting gradient descent with a diminishing learning rate does not work. We first introduce regret with rolling window, a new performance metric for online streaming learning, which measures the performance of an algorithm on every fixed number of contiguous samples. At the same time, we propose a family of algorithms based on gradient descent with a constant or adaptive learning rate and provide very technical analyses establishing regret bound properties of the algorithms. We cover the convex setting showing the regret of the order of the square root of the size of the window in the constant and dynamic learning rate scenarios. Our proof is applicable also to the standard online setting where we provide the first analysis of the same regret order (the previous proofs have flaws). We also study a two layer neural network setting with ReLU activation. In this case we establish that if initial weights are close to a stationary point, the same square root regret bound is attainable. We conduct computational experiments demonstrating a superior performance of the proposed algorithms.

We investigate the use of ellipsoidal trust region constraints for second-order optimization of neural networks. This approach can be seen as a higher-order counterpart of adaptive gradient methods, which we here show to be interpretable as first-order trust region methods with ellipsoidal constraints. In particular, we show that the preconditioning matrix used in RMSProp and Adam satisfies the necessary conditions for convergence of (first- and) second-order trust region methods and report that this ellipsoidal constraint constantly outperforms its spherical counterpart in practice. We furthermore set out to clarify the long-standing question of the potential superiority of Newton methods in deep learning. In this regard, we run extensive benchmarks across different datasets and architectures to find that comparable performance to gradient descent algorithms can be achieved but using Hessian information does not give rise to better limit points and comes at the cost of increased hyperparameter tuning.

The one-class kernel spectral regression (OC-KSR), the regression-based formulation of the kernel null-space approach has been found to be an effective Fisher criterion-based methodology for one-class classification (OCC), achieving state-of-the-art performance in one-class classification while providing relatively high robustness against data corruption. This work extends the OC-KSR methodology to a multi-task setting where multiple one-class problems share information for improved performance. By viewing the multi-task structure learning problem as one of compositional function learning, first, the OC-KSR method is extended to learn multiple tasks' structure \textit{linearly} by posing it as an instantiation of the separable kernel learning problem in a vector-valued reproducing kernel Hilbert space where an output kernel encodes tasks' structure while another kernel captures input similarities. Next, a non-linear structure learning mechanism is proposed which captures multiple tasks' relationships \textit{non-linearly} via an output kernel. The non-linear structure learning method is then extended to a sparse setting where different tasks compete in an output composition mechanism, leading to a sparse non-linear structure among multiple problems. Through extensive experiments on different data sets, the merits of the proposed multi-task kernel null-space techniques are verified against the baseline as well as other existing multi-task one-class learning techniques.

Over-parameterization is ubiquitous nowadays in training neural networks to benefit both optimization in seeking global optima and generalization in reducing prediction error. However, compressive networks are desired in many real world applications and direct training of small networks may be trapped in local optima. In this paper, instead of pruning or distilling an over-parameterized model to compressive ones, we propose a parsimonious learning approach based on differential inclusions of inverse scale spaces, that generates a family of models from simple to complex ones with a better efficiency and interpretability than stochastic gradient descent in exploring the model space. It enjoys a simple discretization, the Split Linearized Bregman Iterations, with provable global convergence that from any initializations, algorithmic iterations converge to a critical point of empirical risks. One may exploit the proposed method to boost the complexity of neural networks progressively. Numerical experiments with MNIST, Cifar-10/100, and ImageNet are conducted to show the method is promising in training large scale models with a favorite interpretability.

Regression, Classification and much more.HOT & NEW 4.8 (7 ratings) 161 students enrolled Created by Denis Panjuta What you'll learn Create machine learning applications in Python as well as R Apply Machine Learning to own data You will learn Machine Learning clearly and concisely Learn with real data: Many practical examples (spam filter, is fungus edible or poisonous etc. ...) No dry mathematics - everything explained vividly Use popular tools like Sklearn, and Caret You will know when to use which machine learning model This course contains over 200 lessons, quizzes, practical examples, ... - the easiest way if you want to learn Machine Learning. Step by step I teach you machine learning. In each section you will learn a new topic - first the idea / intuition behind it, and then the code in both Python and R. Machine Learning is only really fun when you evaluate real data. That's why you analyze a lot of practical examples in this course: Create machine learning applications in Python as well as R Apply Machine Learning to own data You will learn Machine Learning clearly and concisely Learn with real data: Many practical examples (spam filter, is fungus edible or poisonous etc. ...) No dry mathematics - everything explained vividly Use popular tools like Sklearn, and Caret You will know when to use which machine learning model Learn with real data: Many practical examples (spam filter, is fungus edible or poisonous etc. ...)

"You want to ride the wave rather than getting slammed by its disruption. You don't want to be Blockbuster Video or Sears, you want to be Netflix or Amazon." That was how Dave Bluey, assistant professor of practice and career advisor with the Department of Management, explained the reasoning behind the Department of Management's symposium, "How Artificial Intelligence Will Impact Your Career." Over 250 hundred students gathered for a panel discussion led by industry experts to hear about – and in some cases see – the impact artificial intelligence may have on their future careers. The event was a partnership between the Management Department and leading firms in the areas of machine learning, artificial intelligence, and robotics in an on-going Digital Transformation Series at Virginia Tech.