Khare, Shreya
Adversarial Black-Box Attacks On Text Classifiers Using Multi-Objective Genetic Optimization Guided By Deep Networks
Mathai, Alex, Khare, Shreya, Tamilselvam, Srikanth, Mani, Senthil
We propose a novel genetic-algorithm technique that generates black-box adversarial examples which successfully fool neural network based text classifiers. We perform a genetic search with multi-objective optimization guided by deep learning based inferences and Seq2Seq mutation to generate semantically similar but imperceptible adversaries. We compare our approach with DeepWordBug (DWB) on SST and IMDB sentiment datasets by attacking three trained models viz. char-LSTM, word-LSTM and elmo-LSTM. On an average, we achieve an attack success rate of 65.67% for SST and 36.45% for IMDB across the three models showing an improvement of 49.48% and 101% respectively. Furthermore, our qualitative study indicates that 94% of the time, the users were not able to distinguish between an original and adversarial sample.
Democratization of Deep Learning Using DARVIZ
Sankaran, Anush (IBM Research AI) | Panwar, Naveen (IBM Research AI) | Khare, Shreya (IBM Research AI) | Mani, Senthil (IBM Research AI) | Sethi, Akshay (IIIT Delhi) | Aralikatte, Rahul (IBM Research AI) | Gantayat, Neelamadhav (IBM Research AI)
With an abundance of research papers in deep learning, adoption and reproducibility of existing works becomes a challenge. To make a DL developer life easy, we propose a novel system, DARVIZ, to visually design a DL model using a drag-and-drop framework in an platform agnostic manner. The code could be automatically generated in both Caffe and Keras. DARVIZ could import (i) any existing Caffe code, or (ii) a research paper containing a DL design; extract the design, and present it in visual editor.
Hi, How Can I Help You?: Automating Enterprise IT Support Help Desks
Mani, Senthil (IBM Research AI) | Gantayat, Neelamadhav (IBM Research AI) | Aralikatte, Rahul (IBM Research AI) | Gupta, Monika (IBM Research AI) | Dechu, Sampath (IBM Research AI) | Sankaran, Anush (IBM Research AI) | Khare, Shreya (IBM Research AI) | Mitchell, Barry (IBM Global Business Services) | Subramanian, Hemamalini (IBM Global Business Services) | Venkatarangan, Hema (IBM Global Business Services)
Question answering is one of the primary challenges of natural language understanding. In realizing such a system, providing complex long answers to questions is a challenging task as opposed to factoid answering as the former needs context disambiguation. The different methods explored in the literature can be broadly classified into three categories namely: 1) classification based, 2) knowledge graph based and 3) retrieval based. Individually, none of them address the need of an enterprise wide assistance system for an IT support and maintenance domain. In this domain, the variance of answers is large ranging from factoid to structured operating procedures; the knowledge is present across heterogeneous data sources like application specific documentation, ticket management systems and any single technique for a general purpose assistance is unable to scale for such a landscape. To address this, we have built a cognitive platform with capabilities adopted for this domain. Further, we have built a general purpose question answering system leveraging the platform that can be instantiated for multiple products, technologies in the support domain. The system uses a novel hybrid answering model that orchestrates across a deep learning classifier, a knowledge graph based context disambiguation module and a sophisticated bag-of-words search system. This orchestration performs context switching for a provided question and also does a smooth hand-off of the question to a human expert if none of the automated techniques can provide a confident answer. This system has been deployed across 675 internal enterprise IT support and maintenance projects.
Agent Assist: Automating Enterprise IT Support Help Desks
Mani, Senthil (IBM Research AI) | Gantayat, Neelamadhav (IBM Research AI) | Aralikatte, Rahul (IBM Research AI) | Gupta, Monika (IBM Research AI) | Dechu, Sampath (IBM Research AI) | Sankaran, Anush (IBM Research AI) | Khare, Shreya (IBM Research AI) | Mitchell, Barry (IBM Global Business Services) | Subramanian, Hemamalini (IBM Global Business Services) | Venkatarangan, Hema (IBM Global Business Services)
In this paper, we present Agent Assist, a virtual assistant which helps IT support staff to resolve tickets faster. It is essentially a conversation system which provides procedural and often complex answers to queries. This system can ingest knowledge from various sources like application documentation, ticket management systems and knowledge transfer video recordings. It uses an ensemble of techniques like question classification, knowledge graph based disambiguation, information retrieval, etc., to provide quick and relevant solutions to problems from various technical domains and is currently being used in more than 650 projects within IBM.
DLPaper2Code: Auto-Generation of Code From Deep Learning Research Papers
Sethi, Akshay (IIIT Delhi) | Sankaran, Anush (IBM Research AI) | Panwar, Naveen (IBM Research AI) | Khare, Shreya (IBM Research AI) | Mani, Senthil (IBM Research AI)
With an abundance of research papers in deep learning, reproducibility or adoption of the existing works becomes a challenge. This is due to the lack of open source implementations provided by the authors. Even if the source code is available, then re-implementing research papers in a different library is a daunting task. To address these challenges, we propose a novel extensible approach, DLPaper2Code, to extract and understand deep learning design flow diagrams and tables available in a research paper and convert them to an abstract computational graph. The extracted computational graph is then converted into execution ready source code in both Keras and Caffe, in real-time. An arXiv-like website is created where the automatically generated designs is made publicly available for 5,000 research papers. The generated designs could be rated and edited using an intuitive drag-and-drop UI framework in a crowd sourced manner. To evaluate our approach, we create a simulated dataset with over 216,000 valid deep learning design flow diagrams using a manually defined grammar. Experiments on the simulated dataset show that the proposed framework provide more than 93% accuracy in flow diagram content extraction.
DLPaper2Code: Auto-generation of Code from Deep Learning Research Papers
Sethi, Akshay, Sankaran, Anush, Panwar, Naveen, Khare, Shreya, Mani, Senthil
With an abundance of research papers in deep learning, reproducibility or adoption of the existing works becomes a challenge. This is due to the lack of open source implementations provided by the authors. Further, re-implementing research papers in a different library is a daunting task. To address these challenges, we propose a novel extensible approach, DLPaper2Code, to extract and understand deep learning design flow diagrams and tables available in a research paper and convert them to an abstract computational graph. The extracted computational graph is then converted into execution ready source code in both Keras and Caffe, in real-time. An arXiv-like website is created where the automatically generated designs is made publicly available for 5,000 research papers. The generated designs could be rated and edited using an intuitive drag-and-drop UI framework in a crowdsourced manner. To evaluate our approach, we create a simulated dataset with over 216,000 valid design visualizations using a manually defined grammar. Experiments on the simulated dataset show that the proposed framework provide more than $93\%$ accuracy in flow diagram content extraction.