This paper explores automatic detection and classification of exit signs with the aim of enabling a service robot to assist the visually impaired with indoor navigation, inspired by a guide dog. The ultimate aim is to achieve autonomous indoor navigation using computer vision to identify navigational goals in an unfamiliar environment. In particular, we focus on the task of exiting a building by following exit signs that may include arrows that indicate where the next door or sign is located. The proposed method utilizes a deep learning framework, Faster R-CNN, to classify and localize exit signs in real time. The Faster R-CNN model achieved competitive results on more sizable dataset than existing approaches.

Given its easy accessibility and prevalence, Twitter has been actively used as an alternative data source for health surveillance research, and personal health experiences play an important role in such surveillance activities. Therefore, there is a need to develop efficient and effective methods to identify Twitter posts related to personal health experiences. In this work, we present a method which combines word embeddings, convolutional, and Long Short-Term Memory (LSTM) recurrent neural networks to detect personal health experience tweets. The word embedding and convolutional layers serve as a pre-processing step for unsupervised feature learning. This step helps to eliminate the need for feature engineering. We studied three distributed word representation methods: word2vec, fastText, and WordRank to represent the tweet texts in a vector space model. Vectors of the word representations were later used in a convolution layer for further pre-processing, and were fed to an LSTM based Recurrent Neural Network (RNN) model for classification. Our results showed that approach outperforms, with a significant margin, conventional classifiers that used human engineered features. The RNN based model had a significant improvement in precision compared to the other methods (by 123%). This improvement helps to detect more true positive Personal Health Experience tweets.

Towards the vision of translating code that implements an algorithm from one programming language into another, this paper proposes an approach for automated program classification using bilateral tree-based convolutional neural networks (BiTBCNNs). It is layered on top of two tree-based convolutional neural networks (TBCNNs), each of which recognizes the algorithm of code written in an individual programming language. The combination layer of the networks recognizes the similarities and differences among code in different programming languages. The BiTBCNNs are trained using the source code in different languages but known to implement the same algorithms and/or functionalities. For a preliminary evaluation, we use 3591 Java and 3534 C++ code snippets from 6 algorithms we crawled systematically from GitHub. We obtained over 90% accuracy in the cross-language binary classification task to tell whether any given two code snippets implement a same algorithm. Also, for the algorithm classification task, i.e., to predict which one of the six algorithm labels is implemented by an arbitrary C++ code snippet, we achieved over 80% precision.

Due to popularity in texting and messaging, a recent advancement of deep learning technologies, a conversation-based interaction becomes an emerging user interface. While today’s conversation platforms offer basic conversation capabilities such as natural language understanding, entity extraction and simple dialogue management, there are still challenges in developing practical applications to support complex use cases using a dialogue system. In this paper, we highlight such challenges and share practical knowledge learned from our experiences on developing a leisure travel shopping application that combines a personalized recommendation system and a conversation system. Such efforts include a conversation design, extraction of user intents, communication of variables between a dialogue system and analytics engines, and dynamic user interface designs. In particular, we introduce our approach to overcome the unique challenges, understanding user's intent, when dialogue system met personalized recommendation system. Furthermore, we propose a semantic mapping as a novel method to utilize undefined user's preferences when producing recommended items. Finally, examples of recommendations based on natural language conversations are provided in order to exhibit how components in the overall architecture are seamlessly orchestrated. In general, our framework provides guiding principles and best practices on the implementation of task-oriented dialogue system connected with other components in the overall architecture.

We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory. We achieve this by extending unitary RNNs with a gating mechanism. Our model is able to outperform LSTMs, GRUs and Unitary RNNs on several long-term dependency benchmark tasks. We empirically both show the orthogonal/unitary RNNs lack the ability to forget and also the ability of GORU to simultaneously remember long term dependencies while forgetting irrelevant information. This plays an important role in recurrent neural networks. We provide competitive results along with an analysis of our model on many natural sequential tasks including the bAbI Question Answering, TIMIT speech spectrum prediction, Penn TreeBank, and synthetic tasks that involve long-term dependencies such as algorithmic, parenthesis, denoising and copying tasks.

We proposed a deep learning method for interpretable diabetic retinopathy (DR) detection. The visual-interpretable feature of the proposed method is achieved by adding the regression activation map (RAM) after the global averaging pooling layer of the convolutional networks (CNN). With RAM, the proposed model can localize the discriminative regions of an retina image to show the specific region of interest in terms of its severity level. We believe this advantage of the proposed deep learning model is highly desired for DR detection because in practice, users are not only interested with high prediction performance, but also keen to understand the insights of DR detection and why the adopted learning model works. In the experiments conducted on a large scale of retina image dataset, we show that the proposed CNN model can achieve high performance on DR detection compared with the state-of-the-art while achieving the merits of providing the RAM to highlight the salient regions of the input image.

Recent works on gradient-based attacks and universal perturbations can adversarially modify images to bring down the accuracy of state-of-the-art classification techniques based on deep neural networks to as low as 10% on popular datasets like MNIST and ImageNet. The design of general defense strategies against a wide range of such attacks remains a challenging problem. In this paper, we derive inspiration from recent advances in the fields of cybersecurity and multi-agent systems and propose to use the concept of Moving Target Defense (MTD) for increasing the robustness of a set of deep networks against such adversarial attacks. To this end, we formalize and exploit the notion of differential immunity of an ensemble of networks to specific attacks. To classify an input image, a trained network is picked from this set of networks by formulating the interaction between a Defender (who hosts the classification networks) and their (Legitimate and Malicious) Users as a repeated Bayesian Stackelberg Game (BSG).We empirically show that our approach, MTDeep reduces misclassification on perturbed images for MNIST and ImageNet datasets while maintaining high classification accuracy on legitimate test images. Lastly, we demonstrate that our framework can be used in conjunction with any existing defense mechanism to provide more resilience to adversarial attacks than those defense mechanisms by themselves.

Deep learning has been very successful on image classification tasks in the past few years, because it allows to develop end-to-end solutions, taking as input the raw images in form of a grid of pixels and returning the class assignments. Semantic Based Regularization is used in this paper as a general and novel way to integrate prior knowledge into deep learning. Semantic Based Regularization takes as input the prior knowledge, expressed as a collection of first-order logic clauses (FOL), where each task to be learned corresponds to a predicate in the knowledge base. Then, it translates the knowledge into a set of constraints which can be either integrated into the learning process or used in a collective classification step during the test phase. The integration of the domain knowledge during the train or test phase is realized via the same backpropagation schema that runs over the expression trees of the grounded FOL clauses. The methodology can be applied on top of any learner and the experimental results on CIFAR-10 show how the integration of the prior knowledge boosts the accuracy of many different deep architectures.

Automated analysis methods are crucial aids for monitoring and defending a network to protect the sensitive or confidential data it hosts. This work introduces a flexible, powerful, and unsupervised approach to detecting anomalous behavior in computer and network logs; one that largely eliminates domain-dependent feature engineering employed by existing methods. By treating system logs as threads of interleaved ``sentences'' (event log lines) to train online unsupervised neural network language models, our approach provides an adaptive model of normal network behavior. We compare the effectiveness of both standard and bidirectional recurrent neural network language models at detecting malicious activity within network log data. Extending these models, we introduce a tiered recurrent architecture, which provides context by modeling sequences of users' actions over time. Compared to Isolation Forest and Principal Components Analysis, two popular anomaly detection algorithms, we observe superior performance on the Los Alamos National Laboratory Cyber Security dataset. For log-line-level red team detection, our best performing character-based model provides test set area under the receiver operator characteristic curve of 0.98, demonstrating the strong fine-grained anomaly detection performance of this approach on open vocabulary logging sources.

We describe a novel weakly supervised deep learning framework that combines both the discriminative and generative models to learn meaningful representation in the multiple instance learning (MIL) setting. MIL is a weakly supervised learning problem where labels are associated with groups of instances (referred as bags) instead of individual instances. To address the essential challenge in MIL problems raised from the uncertainty of positive instances label, we use a discriminative model regularized by variational autoencoders (VAEs) to maximize the differences between latent representations of all instances and negative instances. As a result, the hidden layer of the variational autoencoder learns meaningful representation. This representation can effectively be used for MIL problems as illustrated by better performance on the standard benchmark datasets comparing to the state-of-the-art approaches. More importantly, unlike most related studies, the proposed framework can be easily scaled to large dataset problems, as illustrated by the audio event detection and segmentation task. Visualization also confirms the effectiveness of the latent representation in discriminating positive and negative classes.