We demonstrate the value of learning dense representations (embeddings) of collections of codes representing various domains ofmo medical information. These embeddings are learned jointly using sparse representations of diagnosis, procedures and prescriptions extracted from medical claims, in order to infer semantic relationships both within, as well as between domains. We show that learning effective embeddings allows for a rich representation of a patient's clinical state at a point in time, a mechanism for assigning robust clinical similarity between patients, and a data representation which is generally useful in modeling various health care related events, such as the next most likely event (i.e. diagnosis, procedure or prescription), or the likelihood of a specific event in the future (e.g. an emergency room visit). Three methods are showcased in this paper including: general embedding, task-specific embedding, and a combination of the two which we have deemed "super" embedding for the purpose of this paper.

In this paper, we present the application of deep learning techniques to develop a modern model for the prediction of graft failure and survival analysis in liver transplant patients. We trained our model using the United Network for Organ Sharing (UNOS) dataset consisting of 59,115 patients from year 2002 to 2016 with around 150 features each. We also compare our model against an- other dataset – Scientific Registry of Transplant Recipients (SRTR) including 87,334 patients from year 2002 to 2018 – after selecting features by mapping them from UNOS data. Some of the most important features common to both datasets are Model for End-stage Liver Disease (MELD) score, patient body mass index (BMI), donor and patient age, cold ischemia time, and levels of various chemicals within the patient. To provide an additional tool to clinical practitioners in the allocation of a scarce resource, we developed a multi-task model to learn the survival function of a donor-recipient pair and hence predict the exact time of failure which outper- forms the traditional cox hazard models. The multi-task model produces very promising C-index results of 0.82 and 0.57 on the SRTR and UNOS datasets respectively.

We explore the performance of multiple maxout activation variants on the big data text sentiment analysis task using convolutional neural networks. Maxout networks have gained great success in many computer vision tasks, but there is limited work on other classification tasks. Our experiments compare ReLU, LReLU, SeLU and tanh to four maxout variants. We evaluate the effectiveness of the activation functions on five datasets, including two datasets collected from the Amazon product reviews corpus, two datasets collected from the Yelp corpus, and the Sentiment140 dataset. Throughout the experiments, we found that maxout networks are slow to train compared to the traditional activation functions. We find that on average across all datasets, ReLU’s classification performance is better than any maxout activation if the number of convolutional filters is doubled. Our experiments suggest that adding more filters enhance the classification accuracy of ReLU, without affecting its comparatively low training time.

Semantic labeling of texts allows people and computing devices to more easily understand the meaning of a natural language sentence as a whole. It is very often one of the steps taken of procedures related to natural language processing. However, this step is often done manually, which is very expensive and time-consuming. When automatic labeling systems are employed, methods such as maximum entropy models are used, which receive as input features specified by specialists that also make the development of the system more expensive. In this article we present a model of the deep recurrent network that semantically annotates texts in English using as labels the top categories of an ontology. The tests showed that it is possible to obtain better results than the models that need the features to be made explicit.

Technical and fundamental analysis are traditional tools used to analyze individual stocks; however, the finance literature has shown that the price movement of each individual stock correlates heavily with other stocks, especially those within the same sector. In this paper we propose a general-purpose market representation that incorporates fundamental and technical indicators and relationships between individual stocks. We treat the daily stock market as a ‘market image’ where rows (grouped by market sector) represent individual stocks and columns represent indicators. We apply a convolutional neural network over this market image to build market features in a hierarchical way. We use a recurrent neural network, with an attention mechanism over the market feature maps, to model temporal dynamics in the market. We show that our proposed model outperforms strong baselines in both short-term and long-term stock return prediction tasks. We also show another use for our market image: to construct concise and dense market embeddings suitable for downstream prediction tasks.

The representation of semantic meaning of sentences using neural network has recently gained popularity, due to the fact that there is no need to specifically extract lexical syntactic and semantic features. A major problem with this approach is that it requires large human annotated corpora. In order to reduce human annotation effort, in recent years, researchers made several attempts to find universal sentence representation methods, aiming to obtain general-purpose sentence embeddings that could be widely adopted to a wide range of NLP tasks without training directly from the specific datasets. InferSent, a supervised universal sentence representation model proposed by Facebook research, implements 8 popular neural network sentence encoding structures trained on natural language inference datasets, and apply to 12 different NLP tasks. However, the relation classification task was not one of these. In this paper, we re-train these 8 sentence encoding structures and use them as the starting points on relation classification task. Experiments using SemEval-2010 datasets show that our models could achieve comparable results to the state-of-the-art relation classification systems.

Thanks to digitization of industrial assets in fleets, the ambitious goal of transferring fault diagnosis models fromone machine to the other has raised great interest. Solving these domain adaptive transfer learning tasks has the potential to save large efforts on manually labeling data and modifying models for new machines in the same fleet. Although data-driven methods have shown great potential in fault diagnosis applications, their ability to generalize on new machines and new working conditions are limited because of their tendency to overfit to the training set in reality. One promising solution to this problem is to use domain adaptation techniques. It aims to improve model performance on the target new machine. Inspired by its successful implementation in computer vision, we introduced Domain-Adversarial Neural Networks (DANN) to our context, along with two other popular methods existing in previous fault diagnosis research. We then carefully justify the applicability of these methods in realistic fault diagnosis settings, and offer a unified experimental protocol for a fair comparison between domain adaptation methods for fault diagnosis problems.

Deep convolutional sum-product networks (DCSPNs) have very recently been introduced and shown to yield state-of-the-art results in image completion tasks. A DCSPN consists of a tree structure (a directed acyclic graph) coupled with parameters of the structure. Given that DCSPNs are in their infancy, many open questions remain regarding the properties and topology of its tree structure. In this paper, we undertake three investigations pertaining to the DCSPN structure. The first two studies revolve around the original structure put forth in the seminal paper. These studies increase the number of pooling layers and vary the hyperparameters in attempts to improve accuracy. The third inquiry suggests a new DCSPN tree structure that significantly lowers the training time at some modest expense of accuracy.

The accuracy of Top-N recommendation task is challenged in the systems with mainly implicit user feedback considered. Adversarial training has presented successful results in identifying real data distributions in various domains (e.g. image processing). Nonetheless, adversarial training applied to recommendation is still challenged especially by interpretation of negative implicit feedback causing it to converge slowly as well as affecting its convergence stability. This is often attributed to high sparsity of the implicit feedback and discrete values characteristic from items recommendation. To face these challenges, we propose a novel model named convolutional adversarial latent factor model (CALF), which uses adversarial training in generative and discriminative models for implicit feedback recommendations. We assume that users prefer observed items over generated items and then apply pairwise product to model the user-item interactions. Additionally, the latent features become input data of our convolutional neural network (CNN) to learn correlations among embedding dimensions. Finally, Rao-Blackwellized sampling is adopted to deal with the discrete values optimizing CALF and stabilizing the training step. We conducted extensive experiments on three different benchmark datasets, where our proposed model demonstrates its efficiency for item recommendation.

Modelling biologically-plausible neural structures for intelligent agents presents a unique challenge when operating in real-time domains. Neurons in our brains have different response properties, firing rates, and propagation lengths, creating noise that cannot be reliably decoded. This research explores the strengths and limitations of LIF spiking neuron ensembles for application in OpenAI virtual environments. Topics discussed include how we represent arbitrary environmental signals from multiple senses, choosing between equally viable actions in a given scenario, and how one can create a generic model that can learn and operate in a verity of situations.