Deep Learning
The Myths of Our Time: Fake News
Růžička, Vít, Kang, Eunsu, Gordon, David, Patel, Ankita, Fashimpaur, Jacqui, Zaheer, Manzil
While the purpose of most fake news is misinformation and political propaganda, our team sees it as a new type of myth that is created by people in the age of internet identities and artificial intelligence. Seeking insights on the fear and desire hidden underneath these modified or generated stories, we use machine learning methods to generate fake articles and present them in the form of an online news blog. This paper aims to share the details of our pipeline and the techniques used for full generation of fake news, from dataset collection to presentation as a media art project on the internet. Keywords: Fake news, Article generation, LSTM, RNN, Language model, Machine learning, AI, Media art, Internet art, Web, Blog, Human-AI Co-Creation Introduction Is fake news a new type of myth that people are creating in the age of internet and artificial intelligence? K. Shu et al says fake news can have many definitions, and one narrow definition is "a news article that is intentionally and verifiably false."
Dimensionality Reduction Flows
Das, Hari Prasanna, Abbeel, Pieter, Spanos, Costas J.
Deep generative modelling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. Trained flow models carry rich information about the structure and local variance in input data. However, a bottleneck for flow models to scale with increasing dimensions is that the latent space has same size as the high-dimensional input space. In this paper, we propose methods to reduce the latent space dimension of flow models. Our first approach includes replacing standard high dimensional prior with a learned prior from a low dimensional noise space. Further improving to achieve exact log-likelihood with reduced dimensionality, our second approach presents an improved multi-scale architecture (Dinh et al., 2016) via likelihood contribution based factorization of dimensions. Using our method over state-of-the-art flow models, we demonstrate improvements in log-likelihood score on standard image benchmarks. Our work ventures a data dependent factorization scheme which is more efficient than static counterparts in prior works.
A principled approach for generating adversarial images under non-smooth dissimilarity metrics
Pooladian, Aram-Alexandre, Finlay, Chris, Hoheisel, Tim, Oberman, Adam
Deep neural networks are vulnerable to adversarial perturbations: small changes in the input easily lead to misclassification. In this work, we propose an attack methodology catered not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarity metric with a closed proximal form. This includes, but is not limited to, $\ell_1$, $\ell_2$, $\ell_\infty$ perturbations, and the $\ell_0$ counting "norm", i.e. true sparseness. Our approach to generating perturbations is a natural extension of our recent work, the LogBarrier attack, which previously required the metric to be differentiable. We demonstrate our new algorithm, ProxLogBarrier, on the MNIST, CIFAR10, and ImageNet-1k datasets. We attack undefended and defended models, and show that our algorithm transfers to various datasets with little parameter tuning. In particular, in the $\ell_0$ case, our algorithm finds significantly smaller perturbations compared to multiple existing methods
Knowledge Isomorphism between Neural Networks
Liang, Ruofan, Li, Tianlin, Li, Longfei, Zhang, Quanshi
This paper aims to analyze knowledge isomorphism between pre-trained deep neural networks. We propose a generic definition for knowledge isomorphism between neural networks at different fuzziness levels, and design a task-agnostic and model-agnostic method to disentangle and quantify isomorphic features from intermediate layers of a neural network. As a generic tool, our method can be broadly used for different applications. In preliminary experiments, we have used knowledge isomorphism as a tool to diagnose feature representations of neural networks. Knowledge isomorphism provides new insights to explain the success of existing deep-learning techniques, such as knowledge distillation and network compression. More crucially, it has been shown that knowledge isomorphism can also be used to refine pre-trained networks and boost performance.
GAN Path Finder: Preliminary results
Soboleva, Natalia, Yakovlev, Konstantin
2D path planning in static environment is a well-known problem and one of the common ways to solve it is to 1) represent the environment as a grid and 2) perform a heuristic search for a path on it. At the same time 2D grid resembles much a digital image, thus an appealing idea comes to being -- to treat the problem as an image generation task and to solve it utilizing the recent advances in deep learning. In this work we make an attempt to apply a generative neural network as a path finder and report preliminary results, convincing enough to claim that this direction of research is worth further exploration.
Learning to Generalize to Unseen Tasks with Bilevel Optimization
Lee, Hayeon, Na, Donghyun, Lee, Hae Beom, Hwang, Sung Ju
Recent metric-based meta-learning approaches, which learn a metric space that generalizes well over combinatorial number of different classification tasks sampled from a task distribution, have been shown to be effective for few-shot classification tasks of unseen classes. They are often trained with episodic training where they iteratively train a common metric space that reduces distance between the class representatives and instances belonging to each class, over large number of episodes with random classes. However, this training is limited in that while the main target is the generalization to the classification of unseen classes during training, there is no explicit consideration of generalization during meta-training phase. To tackle this issue, we propose a simple yet effective meta-learning framework for metricbased approaches, which we refer to as learning to generalize (L2G), that explicitly constrains the learning on a sampled classification task to reduce the classification error on a randomly sampled unseen classification task with a bilevel optimization scheme. This explicit learning aimed toward generalization allows the model to obtain a metric that separates well between unseen classes. We validate our L2G framework on mini-ImageNet and tiered-ImageNet datasets with two base meta-learning few-shot classification models, Prototypical Networks and Relation Networks. The results show that L2G significantly improves the performance of the two methods over episodic training. Further visualization shows that L2G obtains a metric space that clusters and separates unseen classes well.
A Weakly-Supervised Attention-based Visualization Tool for Assessing Political Affiliation
Rajamohan, Srijith, Romanella, Alana, Ramesh, Amit
In this work, we seek to finetune a weakly-supervised expert-guided Deep Neural Network (DNN) for the purpose of determining political affiliations. In this context, stance detection is used for determining political affiliation or ideology which is framed in the form of relative proximities between entities in a low-dimensional space. An attention-based mechanism is used to provide model interpretability. A Deep Neural Network for Natural Language Understanding (NLU) using static and contextual embeddings is trained and evaluated. Various techniques to visualize the projections generated from the network are evaluated for visualization efficiency. An overview of the pipeline from data ingestion, processing and generation of visualization is given here. A web-based framework created to faciliate this interaction and exploration is presented here. Preliminary results of this study are summarized and future work is outlined.
Semantic Role Labeling with Associated Memory Network
Guan, Chaoyu, Cheng, Yuhao, Zhao, Hai
Semantic role labeling (SRL) is a task to recognize all the predicate-argument pairs of a sentence, which has been in a performance improvement bottleneck after a series of latest works were presented. This paper proposes a novel syntax-agnostic SRL model enhanced by the proposed associated memory network (AMN), which makes use of inter-sentence attention of label-known associated sentences as a kind of memory to further enhance dependency-based SRL. In detail, we use sentences and their labels from train dataset as an associated memory cue to help label the target sentence. Furthermore, we compare several associated sentences selecting strategies and label merging methods in AMN to find and utilize the label of associated sentences while attending them. By leveraging the attentive memory from known training data, Our full model reaches state-of-the-art on CoNLL-2009 benchmark datasets for syntax-agnostic setting, showing a new effective research line of SRL enhancement other than exploiting external resources such as well pre-trained language models. 1 Introduction Semantic role labeling (SRL) is a task to recognize all the predicate-argument pairs of a given sentence and its predicates. It is a shallow semantic parsing task, which has been widely used in a series of natural language processing (NLP) tasks, such as information extraction (Liu et al., 2016) and question answering (Abujabal et al., 2017). Generally, SRL is decomposed into four classification subtasks in pipeline systems, consisting of Corresponding author.
N-Shot Learning: Learning More with Less Data
If AI is the new electricity, then data is the new coal. Rapid growth of artificial intelligence and deep learning has the ability to impact countless lives and change the world in ways that, until now, were only dreamed of in comic books. Unfortunately, just as we've seen a hazardous depletion in the amount of available coal, many AI applications have little or no data accessible to them. New technology has made up for a lack of physical resources; likewise, new techniques are needed to allow applications with little data to perform satisfactorily. This is the issue at the heart of what is becoming a very popular field: N-shot Learning. You may be asking, what the heck is a shot, anyway? Fair question.A shot is nothing more than a single example available for training, so in N-shot learning, we have N examples for training.
Quantum Machine Learning and Data Analytics - September 5-6, 2019
With the rapid development of quantum computers, a number of quantum algorithms have been developed and tested on both superconducting qubits based machines and ion trap hardware. Quantum machine learning is expected to be a potential application of quantum computer in the near future. Many quantum machine learning algorithms have been proposed to speed up classical machine learning by quantum computers. At the same time, deep learning has shown great power in solving real world problems. The aim of the workshop is to bring together world leading experts in this new field of quantum machine learning to discuss the recent development of quantum algorithms to perform machine learning tasks on large-scale scientific datasets for various industrial and technological applications and in solving challenging problems in science and engineering.