Since the discovery of adversarial examples in machine learning, researchers have designed several techniques to train neural networks that are robust against different types of attacks (most notably $\ell_\infty$ and $\ell_2$ based attacks). However, it has been observed that the defense mechanisms designed to protect against one type of attack often offer poor performance against the other. In this paper, we introduce Randomized Adversarial Training (RAT), a technique that is efficient both against $\ell_2$ and $\ell_\infty$ attacks. To obtain this result, we build upon adversarial training, a technique that is efficient against $\ell_\infty$ attacks, and demonstrate that adding random noise at training and inference time further improves performance against \ltwo attacks. We then show that RAT is as efficient as adversarial training against $\ell_\infty$ attacks while being robust against strong $\ell_2$ attacks. Our final comparative experiments demonstrate that RAT outperforms all state-of-the-art approaches against $\ell_2$ and $\ell_\infty$ attacks.

This report contains the details regarding our submission to the OffensEval 2019 (SemEval 2019 - Task 6). The competition was based on the Offensive Language Identification Dataset. We first discuss the details of the classifier implemented and the type of input data used and pre-processing performed. We then move onto critically evaluating our performance. We have achieved a macro-average F1-score of 0.76, 0.68, 0.54, respectively for Task a, Task b, and Task c, which we believe reflects on the level of sophistication of the models implemented. Finally, we will be discussing the difficulties encountered and possible improvements for the future.

In autonomous embedded systems, it is often vital to reduce the amount of actions taken in the real world and energy required to learn a policy. Training reinforcement learning agents from high dimensional image representations can be very expensive and time consuming. Autoencoders are deep neural network used to compress high dimensional data such as pixelated images into small latent representations. This compression model is vital to efficiently learn policies, especially when learning on embedded systems. We have implemented this model on the NVIDIA Jetson TX2 embedded GPU, and evaluated the power consumption, throughput, and energy consumption of the autoencoders for various CPU/GPU core combinations, frequencies, and model parameters. Additionally, we have shown the reconstructions generated by the autoencoder to analyze the quality of the generated compressed representation and also the performance of the reinforcement learning agent. Finally, we have presented an assessment of the viability of training these models on embedded systems and their usefulness in developing autonomous policies. Using autoencoders, we were able to achieve 4-5 $\times$ improved performance compared to a baseline RL agent with a convolutional feature extractor, while using less than 2W of power.

One path is conditioned on the latent in many NLP tasks, including language modeling codes, and the other path is conditioned on previously (Bowman et al., 2015; Miao et al., 2016), generated words. KL vanishing happens because dialog response generation (Zhao et al., 2017; (i) the first path can easily get blocked, due Wen et al., 2017), semi-supervised text classification to the lack of good latent codes at the beginning of (Xu et al., 2017), controllable text generation decoder training; (ii) the easiest solution that an (Hu et al., 2017), and text compression (Miao expressive decoder can learn is to ignore the latent and Blunsom, 2016). A prominent component of a code, and relies on the other path only for decoding. VAE is the distribution-based latent representation To remedy this issue, a promising approach is for text sequence observations. This flexible representation to remove the blockage in the first path, and feed allows the VAE to explicitly model holistic meaningful latent codes in training the decoder, so properties of sentences, such as style, topic, and that the decoder can easily adopt them to generate high-level linguistic and semantic features.

Images are comprised of matrices of pixel values. Black and white images are single matrix of pixels, whereas color images have a separate array of pixel values for each color channel, such as red, green, and blue. Pixel values are often unsigned integers in the range between 0 and 255. Although these pixel values can be presented directly to neural network models in their raw format, this can result in challenges during modeling, such as in the slower than expected training of the model. Instead, there can be great benefit in preparing the image pixel values prior to modeling, such as simply scaling pixel values to the range 0-1 to centering and even standardizing the values.

A deep learning model developed by NVIDIA Research turns rough doodles into highly realistic scenes using generative adversarial networks (GANs). The tool is like a smart paintbrush, converting segmentation maps into lifelike images.

Predictive analytics systems are currently one of the most important areas of research and development within the Artificial Intelligence domain and particularly in Machine Learning. One of the "holy grails" of predictive analytics is the research and development of the "perfect" recommendation system. In our paper we propose an advanced pipeline model for the multi-task objective of determining product complementarity, similarity and sales prediction using deep neural models applied to big-data sequential transaction systems. Our highly parallelized hybrid pipeline consists of both unsupervised and supervised models, used for the objectives of generating semantic product embeddings and predicting sales, respectively. Our experimentation and benchmarking have been done using very large pharma-industry retailer Big Data stream.

Magnetic resonance imaging (MRI) is widely used in clinical practice for visualizing both biological structure and function, but its use has been traditionally limited by its slow data acquisition. Recent advances in compressed sensing (CS) techniques for MRI that exploit sparsity models of images reduce acquisition time while maintaining high image quality. Whereas classical CS assumes the images are sparse in a known analytical dictionary or transform domain, methods that use learned image models for reconstruction have become popular in recent years. The model could be learned from a dataset and used for reconstruction or learned simultaneously with the reconstruction, a technique called blind CS (BCS). While the well-known synthesis dictionary model has been exploited for MRI reconstruction, recent advances in transform learning (TL) provide an efficient alternative framework for sparse modeling in MRI. TL-based methods enjoy numerous advantages including exact sparse coding, transform update, and clustering solutions, cheap computation, and convergence guarantees, and provide high quality results in MRI as well as in other inverse problems compared to popular competing methods. This paper provides a review of key works in MRI reconstruction from limited data, with focus on the recent class of TL-based reconstruction methods. A unified framework for incorporating various TL-based models is presented. We discuss the connections between transform learning and convolutional or filterbank models and corresponding multi-layer extensions, as well as connections to unsupervised and supervised deep learning. Finally, we discuss recent trends in MRI, open problems, and future directions for the field.

Extreme multi-label text classification (XMTC) addresses the problem of tagging each text with the most relevant labels from an extreme-scale label set. Traditional methods use bag-of-words (BOW) representations without context information as their features. The state-ot-the-art deep learning-based method, AttentionXML, which uses a recurrent neural network (RNN) and the multi-label attention, can hardly deal with extreme-scale (hundreds of thousands labels) problem. To address this, we propose our HAXMLNet, which uses an efficient and effective hierarchical structure with the multi-label attention. Experimental results show that HAXMLNet reaches a competitive performance with other state-of-the-art methods.

In this work, we study abstractive text summarization by exploring different models such as LSTM-encoder-decoder with attention, pointer-generator networks, coverage mechanisms, and transformers. Upon extensive and careful hyperparameter tuning we compare the proposed architectures against each other for the abstractive text summarization task. Finally, as an extension of our work, we apply our text summarization model as a feature extractor for a fake news detection task where the news articles prior to classification will be summarized and the results are compared against the classification using only the original news text.