Deep Learning
Beyond Adversarial Training: Min-Max Optimization in Adversarial Attack and Defense
Wang, Jingkang, Zhang, Tianyun, Liu, Sijia, Chen, Pin-Yu, Xu, Jiacen, Fardad, Makan, Li, Bo
The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations. Nonetheless, min-max optimization beyond the purpose of AT has not been rigorously explored in the research of adversarial attack and defense. In particular, given a set of risk sources (domains), minimizing the maximal loss induced from the domain set can be reformulated as a general min-max problem that is different from AT, since the maximization is taken over the probability simplex of the domain set. Examples of this general formulation include attacking model ensembles, devising universal perturbation to input samples or data transformations, and generalized AT over multiple norm-ball threat models. We show that these problems can be solved under a unified and theoretically principled min-max optimization framework. Our proposed approach leads to substantial performance improvement over the uniform averaging strategy in four different tasks. Moreover, we show how the self-adjusted weighting factors of the probability simplex from our proposed algorithms can be used to explain the importance of different attack and defense models.
There is no general AI: Why Turing machines cannot pass the Turing test
Since 1950, when Alan Turing proposed what has since come to be called the Turing test, the ability of a machine to pass this test has established itself as the primary hallmark of general AI. To pass the test, a machine would have to be able to engage in dialogue in such a way that a human interrogator could not distinguish its behaviour from that of a human being. AI researchers have attempted to build machines that could meet this requirement, but they have so far failed. To pass the test, a machine would have to meet two conditions: (i) react appropriately to the variance in human dialogue and (ii) display a human-like personality and intentions. We argue, first, that it is for mathematical reasons impossible to program a machine which can master the enormously complex and constantly evolving pattern of variance which human dialogues contain. And second, that we do not know how to make machines that possess personality and intentions of the sort we find in humans. Since a Turing machine cannot master human dialogue behaviour, we conclude that a Turing machine also cannot possess what is called ``general'' Artificial Intelligence. We do, however, acknowledge the potential of Turing machines to master dialogue behaviour in highly restricted contexts, where what is called ``narrow'' AI can still be of considerable utility.
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Chen, Wenhu, Chen, Jianshu, Qin, Pengda, Yan, Xifeng, Wang, William Yang
Semantically controlled neural response generation on limited-domain has achieved great performance. However, moving towards multi-domain large-scale scenarios are shown to be difficult because the possible combinations of semantic inputs grow exponentially with the number of domains. To alleviate such scalability issue, we exploit the structure of dialog acts to build a multi-layer hierarchical graph, where each act is represented as a root-to-leaf route on the graph. Then, we incorporate such graph structure prior as an inductive bias to build a hierarchical disentangled self-attention network, where we disentangle attention heads to model designated nodes on the dialog act graph. By activating different (disentangled) heads at each layer, combinatorially many dialog act semantics can be modeled to control the neural response generation. On the large-scale Multi-Domain-WOZ dataset, our model can yield a significant improvement over the baselines on various automatic and human evaluation metrics.
Selection Bias Explorations and Debias Methods for Natural Language Sentence Matching Datasets
Zhang, Guanhua, Bai, Bing, Liang, Jian, Bai, Kun, Chang, Shiyu, Yu, Mo, Zhu, Conghui, Zhao, Tiejun
Natural Language Sentence Matching (NLSM) has gained substantial attention from both academics and the industry, and rich public datasets contribute a lot to this process. However, biased datasets can also hurt the generalization performance of trained models and give untrustworthy evaluation results. For many NLSM datasets, the providers select some pairs of sentences into the datasets, and this sampling procedure can easily bring unintended pattern, i.e., selection bias. One example is the QuoraQP dataset, where some content-independent naive features are unreasonably predictive. Such features are the reflection of the selection bias and termed as the leakage features. In this paper, we investigate the problem of selection bias on six NLSM datasets and find that four out of them are significantly biased. We further propose a training and evaluation framework to alleviate the bias. Experimental results on QuoraQP suggest that the proposed framework can improve the generalization ability of trained models, and give more trustworthy evaluation results for real-world adoptions.
Facial Expression Recognition on FIFA videos using Deep Learning: World Cup Edition
Few Hearts were broken few still live. No matter who wins, the game will still make me thrill. Fifa world cup 2018 has become one of the highest goal scoring world cups in history. No matter which country is playing, the moment those 11 players step on the field, people get connected to them emotionally. While watching them we share their joy, fear, and excitement through the expression conveyed by them.
Developing a NLP based PR platform for the Canadian Elections
Elections are a vital part of democracy allowing people to vote for the candidate they think can best lead the country. A candidate's campaign aims to demonstrate to the public why they think they are the best choice. However, in this age of constant media coverage and digital communications, the candidate is scrutinized at every step. A single misquote or negative news about a candidate can be the difference between him winning or losing the election. It becomes crucial to have a public relations manager who can guide and direct the candidate's campaign by prioritizing specific campaign activities. One critical aspect of the PR manager's work is to understand the public perception of their candidate and improve public sentiment about the candidate.
Introduction to Artificial Intelligence โ Journey of Analytics
Lately I've been exploring deep learning algorithms, and automating system with Artificial Intelligence. Plus, I received a couple of emails asking me about programming skills for AI. So, with those questions in mind, here is a simple introduction to artificial intelligence. AI or artificial intelligence is the process of using software to perform human tasks. It is considered to be a branch of machine learning, and sophisticated algorithms are used to do everything from automating repetitive tasks to creating self-learning sentient systems.
The quest for AI creativity
AI's role in Morgan, and numerous other creative endeavors, shows how far AI has come. Using techniques such as deep learning has enabled tremendous progress, but AI remains relegated to an assistant role--for now. "What's interesting is that, compared to a lot of other machine learning techniques, deep learning technology is what's called a'generative model,' meaning that it learns how to mimic the data it's been trained on," explains Jason Toy, CEO of Somatic, a start-up focused on developing deep learning applications. "If you feed it thousands of paintings and pictures, all of a sudden you have this mathematical system where you can tweak the parameters or the vectors and get brand new creative things similar to what it was trained on." But even highly touted AI techniques have their limitations.
Synthetic Data Is A Tool For Improving Training And Accuracy Of Deep Learning Systems
Data has always been a critical requirement for computer systems. Without enough data, testing is not robust. What's important in artificial intelligence applications using deep learning (DL) is not the volume of data for the sake of volume, it's variety. For instance, in training autonomous vehicle systems, there are an unending set of driving scenarios. In facial recognition, too many systems have yet to train with a wide variety of faces.
Twitter acquires Deep Learning Startup, Fabula AI
Twitter announced that it has acquired London-based Fabula AI. The financial terms of transactions are not disclosed. The announcement stated that Twitter has established a research group lead by Sandeep Pandey. The research groups look into areas like natural language processing, reinforcement learning, ML ethics, recommendation systems, and graph deep learning. In one of the posts titled "Fake News revealed through artificial intelligence", it was revealed that Fabula AI team, Michael Bronstein, professor and researcher at the USI Institute of Computational Science (ICS), fellow ICS researchers Federico Monti and Dr Davide Eynard, developed a new method based on algorithms and artificial intelligence that could prove to be the most effective solution to the spreading of fake news through the Internet.