Deep Learning
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, Alexis, Lample, Guillaume, Rinott, Ruty, Williams, Adina, Bowman, Samuel R., Schwenk, Holger, Stoyanov, Veselin
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in cross-lingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best performance among available baselines.
Part-based Graph Convolutional Network for Action Recognition
Thakkar, Kalpit, Narayanan, P J
Human actions comprise of joint motion of articulated body parts or "gestures". Human skeleton is intuitively represented as a sparse graph with joints as nodes and natural connections between them as edges. Graph convolutional networks have been used to recognize actions from skeletal videos. We introduce a part-based graph convolutional network (PB-GCN) for this task, inspired by Deformable Part-based Models (DPMs). We divide the skeleton graph into four subgraphs with joints shared across them and learn a recognition model using a part-based graph convolutional network. We show that such a model improves performance of recognition, compared to a model using entire skeleton graph. Instead of using 3D joint coordinates as node features, we show that using relative coordinates and temporal displacements boosts performance. Our model achieves state-of-the-art performance on two challenging benchmark datasets NTURGB D and HDM05, for skeletal action recognition.
Focus Group on Artificial Intelligence for Health
Salathรฉ, Marcel, Wiegand, Thomas, Wenzel, Markus
Artificial Intelligence (AI) - the phenomenon of machines being able to solve problems that require human intelligence - has in the past decade seen an enormous rise of interest due to significant advances in effectiveness and use. The health sector, one of the most important sectors for societies and economies worldwide, is particularly interesting for AI applications, given the ongoing digitalisation of all types of health information. The potential for AI assistance in the health domain is immense, because AI can support medical decision making at reduced costs, everywhere. However, due to the complexity of AI algorithms, it is difficult to distinguish good from bad AI-based solutions and to understand their strengths and weaknesses, which is crucial for clarifying responsibilities and for building trust. For this reason, the International Telecommunication Union (ITU) has established a new Focus Group on "Artificial Intelligence for Health" (FG-AI4H) in partnership with the World Health Organization (WHO). Health and care services are usually the responsibility of a government - even when provided through private insurance systems - and thus under the responsibility of WHO/ITU member states. FG-AI4H will identify opportunities for international standardization, which will foster the application of AI to health issues on a global scale. In particular, it will establish a standardized assessment framework with open benchmarks for the evaluation of AI-based methods for health, such as AI-based diagnosis, triage or treatment decisions.
Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection
Qian, Jing, ElSherief, Mai, Belding, Elizabeth M., Wang, William Yang
Hate speech detection is a critical, yet challenging problem in Natural Language Processing (NLP). Despite the existence of numerous studies dedicated to the development of NLP hate speech detection approaches, the accuracy is still poor. The central problem is that social media posts are short and noisy, and most existing hate speech detection solutions take each post as an isolated input instance, which is likely to yield high false positive and negative rates. In this paper, we radically improve automated hate speech detection by presenting a novel model that leverages intra-user and inter-user representation learning for robust hate speech detection on Twitter. In addition to the target Tweet, we collect and analyze the user's historical posts to model intra-user Tweet representations. To suppress the noise in a single Tweet, we also model the similar Tweets posted by all other users with reinforced inter-user representation learning techniques. Experimentally, we show that leveraging these two representations can significantly improve the f-score of a strong bidirectional LSTM baseline model by 10.1%.
Building Brains: How Pearson Plans To Automate Education With AI
On a balmy summer's day in San Francisco, Milena Marinova is sitting on the roof terrace of the offices of Pearson, a company in the midst of a radical transformation from publishing powerhouse to digital-education platform, wrapped in a gray shawl and explaining how she plans to build advanced, deep-learning algorithms that could educate the next generation of students. This is no easy task. With millions of students using its education-software, Pearson has amassed "terrabytes" of data from student homework and even textbooks that have been digitized, data that Marinova is now pulling together to build software that can automatically give students feedback on their work like a teacher would. Instead of just telling them that an answer is right or wrong, a future update to Pearson's math homework tool will give more detailed feedback on how they went wrong in the steps taken to get an answer, Marinova told Forbes in an interview. Pearson is starting with math because the topic is relatively easy to structure and digitize.
13 Deep Learning Frameworks for Natural Language Processing in Python
Chainer, developed by the Japanese company Preferred Networks founded in 2014, is a powerful, flexible, and intuitive Python-based framework for neural networks that adopts a "define-by-run" scheme [1]. It stores the history of computation instead of programming logic. The framework released under the MIT License and is already applied for sentiment analysis, machine translation, speech recognition, question answering, and so on using different types of neural networks like convolutional networks, recurrent networks, and sequence to sequence models [2]. Deeplearning4j is a deep learning Java programming library, but it also has a Python API, Keras that will be described below. Distributed CPUs and GPUs, parallel training via iterative reduce, and micro-service architecture adaptation are its main features [3].
The tremendous potential of Machine Learning in satellite imagery
With the popularization of Artificial Intelligence and its gradual emergence as the core technology that is impelling momentous developments in a large number of fields, there has been a spurt in the use of machine learning and deep learning as well. As per multiple surveys and studies, AI and Machine Learning would be among the highest-paid and most lucrative career streams in the years to come. AI and Machine Learning would revolutionize our existing technological frameworks and usher in a new industrial age by reorienting and transforming everything from the simplest of appliances to automobiles. The applications of Machine Learning are not only limited to the terrestrial zone but have reached for the sky too, both literally as well as figuratively. Just like all other domains that are constantly reimagining themselves and girding for the future, the domain of remote sensing is also undergoing profound changes and witnessing increasing use of specified algorithms when Big Data and Cloud have become almost ubiquitous.
Unity tweaks AI training tools, makes bid for academic respect
Unity Technologies on Monday released version 0.5 of its ML-Agents toolkit to make its Unity 3D game development platform better suited for developing and training autonomous agent code via machine learning. Initially rolled out a year ago in beta, version 0.5 comes with a few improvements. There's a wrapper for Gym (a toolkit for developing and testing reinforcement learning algorithms), support for letting agents make multiple action selections at once and for preventing agents from taking certain actions, and a refurbished set of environments called Marathon Environments. In these virtual spaces, AI researchers can teach software agents to perform certain tasks by rewarding them for correct actions. This sort of reinforcement learning can be limited to digital environments like video games or mapped to software-driven machines in the real world. Through its latest code update, Unity is making the case for Unity 3D as a key tool for AI research, a goal that company code boffins describe in a preprint paper titled, "Unity: A General Platform for Intelligent Agents."
Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
Li, Dan, Chen, Dacheng, Goh, Jonathan, Ng, See-kiong
Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with networked sensors and actuators that are targets for cyber-attacks. Conventional detection techniques are unable to deal with the increasingly dynamic and complex nature of the CPSs. On the other hand, the networked sensors and actuators generate large amounts of data streams that can be continuously monitored for intrusion events. Unsupervised machine learning techniques can be used to model the system behaviour and classify deviant behaviours as possible attacks. In this work, we proposed a novel Generative Adversarial Networks-based Anomaly Detection (GAN-AD) method for such complex networked CPSs. We used LSTM-RNN in our GAN to capture the distribution of the multivariate time series of the sensors and actuators under normal working conditions of a CPS. Instead of treating each sensor's and actuator's time series independently, we model the time series of multiple sensors and actuators in the CPS concurrently to take into account of potential latent interactions between them. To exploit both the generator and the discriminator of our GAN, we deployed the GAN-trained discriminator together with the residuals between generator-reconstructed data and the actual samples to detect possible anomalies in the complex CPS. We used our GAN-AD to distinguish abnormal attacked situations from normal working conditions for a complex six-stage Secure Water Treatment (SWaT) system. Experimental results showed that the proposed strategy is effective in identifying anomalies caused by various attacks with high detection rate and low false positive rate as compared to existing methods.
Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors
Shafaei, Alireza, Schmidt, Mark, Little, James J.
In the real world, a learning system could receive an input that looks nothing like anything it has seen during training, and this can lead to unpredictable behaviour. We thus need to know whether any given input belongs to the population distribution of the training data to prevent unpredictable behaviour in deployed systems. A recent surge of interest on this problem has led to the development of sophisticated techniques in the deep learning literature. However, due to the absence of a standardized problem formulation or an exhaustive evaluation, it is not evident if we can rely on these methods in practice. What makes this problem different from a typical supervised learning setting is that we cannot model the diversity of out-of-distribution samples in practice. The distribution of outliers used in training may not be the same as the distribution of outliers encountered in the application. Therefore, classical approaches that learn inliers vs. outliers with only two datasets can yield optimistic results. We introduce OD-test, a three-dataset evaluation scheme as a practical and more reliable strategy to assess progress on this problem. The OD-test benchmark provides a straightforward means of comparison for methods that address the out-of-distribution sample detection problem. We present an exhaustive evaluation of a broad set of methods from related areas on image classification tasks. Furthermore, we show that for realistic applications of high-dimensional images, the existing methods have low accuracy. Our analysis reveals areas of strength and weakness of each method.