Goto

Collaborating Authors

 Deep Learning


Deep UL2DL: Channel Knowledge Transfer from Uplink to Downlink

arXiv.org Machine Learning

Knowledge of the channel state information (CSI) at the transmitter side is one of the primary sources of information that can be used for efficient allocation of wireless resources. Obtaining Down-Link (DL) CSI in Frequency Division Duplexing (FDD) systems from Up-Link (UL) CSI is not as straightforward as in TDD systems, and so usually users feedback the DL-CSI to the transmitter. To remove the need for feedback (and thus having less signaling overhead), several methods have been studied to estimate DL-CSI from UL-CSI. In this paper, we propose a scheme to infer DL-CSI by observing UL-CSI in which we use two recent deep neural network structures: a) Convolutional Neural networks and b) Generative Adversarial Networks. The proposed deep network structures first learn a latent model of the environment from the training data. Then, the result latent model is used to predict the DL-CSI from the UL-CSI. We have simulated the proposed scheme and evaluated its performance in a few network settings. Simulation results (for different multipath environments) demonstrate efficiency of both direct and generative approaches for UL2DL prediction. One key feature of new generation of cellular networks is their efficient use of frequency bands and energy. To achieve this goal, they use various techniques such as water-filling, appropriate precoding and beamforming. In Time Division Duplexing (TDD) systems, Up-Link (UL) and Down-Link (DL) frequencies are equal, so we can use channel reciprocity and simply infer the DL channel by observing the UL channel.


A Multi-level Neural Network for Implicit Causality Detection in Web Texts

arXiv.org Artificial Intelligence

Abstract--Mining causality from text is a complex and crucial natural language understanding task. Most of the early attempts at its solution can group into two categories: 1) utilizing co-occurrence frequency and world knowledge for causality detection; 2) extracting cause-effect pairs by using connectives and syntax patterns directly. However, because causality has various linguistic expressions, the noisy data and ignoring implicit expressions problems induced by these methods cannot be avoided. In this paper, we present a neural causality detection model, namely Multilevel Causality Detection Network (MCDN), to address this problem. Specifically, we adopt multi-head self-attention to acquire semantic feature at word level and integrate a novel Relation Network to infer causality at segment level. To the best of our knowledge, in touch with the causality tasks, this is the first time that the Relation Network is applied. The experimental results on the AltLex dataset, demonstrate that: a) MCDN is highly effective for the ambiguous and implicit causality inference; b) comparing with the regular text classification task, causality detection requires stronger inference capability; c) the proposed approach achieved state-of- the-art performance. I. Introduction Automatic text causality mining is a critical but difficult task because causality is thought to play an essential role in human cognition when making decisions [1]. Thus, automatic text causality has been studied extensively in a wide range of areas, such as industry [2], physics [3] and healthcare [4], etc. A tool to automatically scour the plethora of textual content on the web and extract meaningful causal relations could help us construct causal chains to unveil previously unknown relationships between events [5] and accelerates the discovery of the intrinsic logic of the events [6]. Many research efforts have been made to mine causality from text corpus with complex sentence structures in the books or newspapers. In Causal-TimeBank [7] authors introduced "CLINK" and "C-SIGNAL" tag to mark events causal relation and causal signals respectively based on specific templates (e.g., "A happened because of B").


RefNet: A Reference-aware Network for Background Based Conversation

arXiv.org Artificial Intelligence

Existing conversational systems tend to generate generic responses. Recently, Background Based Conversations (BBCs) have been introduced to address this issue. Here, the generated responses are grounded in some background information. The proposed methods for BBCs are able to generate more informative responses, they either cannot generate natural responses or have difficulty in locating the right background information. In this paper, we propose a Reference-aware Network (RefNet) to address the two issues. Unlike existing methods that generate responses token by token, RefNet incorporates a novel reference decoder that provides an alternative way to learn to directly cite a semantic unit (e.g., a span containing complete semantic information) from the background. Experimental results show that RefNet significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, indicating that RefNet can generate more appropriate and human-like responses.


VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation

arXiv.org Artificial Intelligence

In this paper, we show how novel transfer reinforcement learning techniques can be applied to the complex task of target driven navigation using the photorealistic AI2THOR simulator. Specifically, we build on the concept of Universal Successor Features with an A3C agent. We introduce the novel architectural contribution of a Successor Feature Dependant Policy (SFDP) and adopt the concept of Variational Information Bottlenecks to achieve state of the art performance. VUSFA, our final architecture, is a straightforward approach that can be implemented using our open source repository. Our approach is generalizable, showed greater stability in training, and outperformed recent approaches in terms of transfer learning ability.


Three Invaluable Ways AI and Neuroscience Are Driving Each Other Forward

#artificialintelligence

DeepMind's Demis Hassabis once pointed to the human brain as a paramount inspiration for building AI with human-like intelligence. The meteoric success of deep learning showcases how insights from neuroscience--memory, learning, decision-making, vision--can be distilled into algorithms that bestow silicon minds with a shadow of our cognitive prowess. This month, the prestigious journal Nature published an entire series highlighting the symbiotic growth between neuroscience and AI. It's been a long time coming. At their core, both disciplines are solving the same central problem--intelligence--but coming from different angles, and at different levels of abstraction.


How to Become More Marketable as a Data Scientist

#artificialintelligence

This headline may seem a bit odd to you. Since data science has a huge impact on today's businesses, the demand for DS experts is growing. At the moment I'm writing this, there are 144,527 data science jobs on LinkedIn alone. But still, it's important to keep your finger on the pulse of the industry to be aware of the fastest and most efficient data science solutions. To help you out, our data-obsessed CV Compiler team analyzed some vacancies and defined the data science employment trends of 2019.


EigenRank by Committee: A Data Subset Selection and Failure Prediction paradigm for Robust Deep Learning based Medical Image Segmentation

arXiv.org Machine Learning

-- Translation of fully automated deep learning based medical image segmentation technologies to clinical workflows face two main algorithmic challenges. The first, is the collection and archival of large quantities of manually annotated ground truth data for both training and validation. The second is the relative inability of the majority of deep learning based segmentation techniques to alert physicians to a likely segmentation failure. Here we propose a novel algorithm, named'Eigenrank' which addresses both of these challenges. Eigenrank can select for manual labeling, a subset of medical images from a large database, such that a U-Net trained on this subset is superior to one trained on a randomly selected subset of the same size. Eigenrank can also be used to pick out, cases in a large database, where deep learning segmentation will fail. We present our algorithm, followed by results and a discussion of how Eigenrank exploits the V on Neumann information to perform both data subset selection and failure prediction for medical image segmentation using deep learning. I. INTRODUCTION A. Significance Deep learning methods have become the mainstay of fully automatic medical image segmentation. These methods play a key role in the development of quantitative imaging biomarkers for a number of pathologies. However, training and deploying deep learning segmentation in practice is beset by a number of challenges. Failure Prediction (FP) - the ability to predict on which cases a deep learning based segmentation model will fail. Both problems are significant in medical image segmentation, more than natural image segmentation. This is because, availability of expert annotated data for training medical image segmentation models is severely constrained.


Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics

arXiv.org Machine Learning

A good feature representation is a determinant factor to achieve high performance for many machine learning algorithms in terms of classification. This is especially true for techniques that do not build complex internal representations of data (e.g. decision trees, in contrast to deep neural networks). To transform the feature space, feature construction techniques build new high-level features from the original ones. Among these techniques, Genetic Programming is a good candidate to provide interpretable features required for data analysis in high energy physics. Classically, original features or higher-level features based on physics first principles are used as inputs for training. However, physicists would benefit from an automatic and interpretable feature construction for the classification of particle collision events. Our main contribution consists in combining different aspects of Genetic Programming and applying them to feature construction for experimental physics. In particular, to be applicable to physics, dimensional consistency is enforced using grammars. Results of experiments on three physics datasets show that the constructed features can bring a significant gain to the classification accuracy. To the best of our knowledge, it is the first time a method is proposed for interpretable feature construction with units of measurement, and that experts in high-energy physics validate the overall approach as well as the interpretability of the built features.


Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings

arXiv.org Machine Learning

--Traditionally, many text-mining tasks treat individual word-tokens as the finest meaningful semantic granularity. However, in many languages and specialized corpora, words are composed by concatenating semantically meaningful subword structures. With regard to word embedding techniques, this leads to not only poor embeddings for infrequent words in long-tailed text corpora but also weak capabilities for handling out-of-vocabulary words. In this paper we propose MorphMine for unsupervised morpheme segmentation. MorphMine applies a parsimony criterion to hierarchically segment words into the fewest number of morphemes at each level of the hierarchy. This leads to longer shared morphemes at each level of segmentation. Experiments show that MorphMine segments words in a variety of languages into human-verified morphemes. Additionally, we experimentally demonstrate that utilizing MorphMine morphemes to enrich word embeddings consistently improves embedding quality on a variety of of embedding evaluations and a downstream language modeling task. I NTRODUCTION Decomposing individual words into finer-granularity morphemes is a necessary step for automatically preprocessing concatenative vocabularies where the number of unique word forms is very large. While linguistic approaches can be used to tackle such segmentation, such rule-based approaches are often tailored to specific languages or domains. As such, data-driven, unsupervised methods that forgo linguistic knowledge have been studied [7], [18]. Typically, these methods focus on segmenting words by applying a probabilistic model or compression algorithms to a full text corpus.


Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation

arXiv.org Machine Learning

Deep neural networks are known to be fragile to small adversarial perturbations. This issue becomes more critical when a neural network is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on neural network certification tools (which are mainly used in static settings such as image classification) with robust control theory to certify a neural network policy in a control loop. Specifically, we give a sufficient condition and an algorithm to ensure that the closed loop state and control constraints are satisfied when the persistent adversarial perturbation is l-infinity norm bounded. Our method is based on finding a positively invariant set of the closed loop dynamical system, and thus we do not require the differentiability or the continuity of the neural network policy. Along with the verification result, we also develop an effective attack strategy for neural network control systems that outperforms exhaustive Monte-Carlo search significantly. We show that our certification algorithm works well on learned models and achieves 5 times better result than the traditional Lipschitz-based method to certify the robustness of a neural network policy on a cart pole control problem.