Deep Learning
Deep Boosted Regression for MR to CT Synthesis
Kläser, Kerstin, Markiewicz, Pawel, Ranzini, Marta, Li, Wenqi, Modat, Marc, Hutton, Brian F, Atkinson, David, Thielemans, Kris, Cardoso, M Jorge, Ourselin, Sebastien
Attenuation correction is an essential requirement of positron emission tomography (PET) image reconstruction to allow for accurate quantification. However, attenuation correction is particularly challenging for PET-MRI as neither PET nor magnetic resonance imaging (MRI) can directly image tissue attenuation properties. MRI-based computed tomography (CT) synthesis has been proposed as an alternative to physics based and segmentation-based approaches that assign a population-based tissue density value in order to generate an attenuation map. We propose a novel deep fully convolutional neural network that generates synthetic CTs in a recursive manner by gradually reducing the residuals of the previous network, increasing the overall accuracy and generalisability, while keeping the number of trainable parameters within reasonable limits. The model is trained on a database of 20 pre-acquired MRI/CT pairs and a four-fold random bootstrapped validation with a 80:20 split is performed. Quantitative results show that the proposed framework outperforms a state-of-the-art atlas-based approach decreasing the Mean Absolute Error (MAE) from 131HU to 68HU for the synthetic CTs and reducing the PET reconstruction error from 14.3% to 7.2%.
Learning to Support: Exploiting Structure Information in Support Sets for One-Shot Learning
Liu, Jinchao, Gibson, Stuart J., Osadchy, Margarita
Deep Learning shows very good performance when trained on large labeled data sets. The problem of training a deep net on a few or one sample per class requires a different learning approach which can generalize to unseen classes using only a few representatives of these classes. This problem has previously been approached by meta-learning. Here we propose a novel meta-learner which shows state-of-the-art performance on common benchmarks for one/few shot classification. Our model features three novel components: First is a feed-forward embedding that takes random class support samples (after a customary CNN embedding) and transfers them to a better class representation in terms of a classification problem. Second is a novel attention mechanism, inspired by competitive learning, which causes class representatives to compete with each other to become a temporary class prototype with respect to the query point. This mechanism allows switching between representatives depending on the position of the query point. Once a prototype is chosen for each class, the predicated label is computed using a simple attention mechanism over prototypes of all considered classes. The third feature is the ability of our meta-learner to incorporate deeper CNN embedding, enabling larger capacity. Finally, to ease the training procedure and reduce overfitting, we averages the top $t$ models (evaluated on the validation) over the optimization trajectory. We show that this approach can be viewed as an approximation to an ensemble, which saves the factor of $t$ in training and test times and the factor of of $t$ in the storage of the final model.
Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents
Zhou, Ganbin, Cao, Rongyu, Ao, Xiang, Luo, Ping, Lin, Fen, Lin, Leyu, He, Qing
In this study, we focus on extracting knowledgeable snippets and annotating knowledgeable documents from Web corpus, consisting of the documents from social media and We-media. Informally, knowledgeable snippets refer to the text describing concepts, properties of entities, or relations among entities, while knowledgeable documents are the ones with enough knowledgeable snippets. These knowledgeable snippets and documents could be helpful in multiple applications, such as knowledge base construction and knowledge-oriented service. Previous studies extracted the knowledgeable snippets using the pattern-based method. Here, we propose the semantic-based method for this task. Specifically, a CNN based model is developed to extract knowledgeable snippets and annotate knowledgeable documents simultaneously. Additionally, a "low-level sharing, high-level splitting" structure of CNN is designed to handle the documents from different content domains. Compared with building multiple domain-specific CNNs, this joint model not only critically saves the training time, but also improves the prediction accuracy visibly. The superiority of the proposed method is demonstrated in a real dataset from Wechat public platform.
Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model
Xu, Kun, Wu, Lingfei, Wang, Zhiguo, Yu, Mo, Chen, Liwei, Sheinin, Vadim
Existing neural semantic parsers mainly utilize a sequence encoder, i.e., a sequential LSTM, to extract word order features while neglecting other valuable syntactic information such as dependency graph or constituent trees. In this paper, we first propose to use the \textit{syntactic graph} to represent three types of syntactic information, i.e., word order, dependency and constituency features. We further employ a graph-to-sequence model to encode the syntactic graph and decode a logical form. Experimental results on benchmark datasets show that our model is comparable to the state-of-the-art on Jobs640, ATIS and Geo880. Experimental results on adversarial examples demonstrate the robustness of the model is also improved by encoding more syntactic information.
Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding
Yoon, Deunsol, Lee, Dongbok, Lee, SangKeun
In this paper, we propose Dynamic Self-Attention (DSA), a new self-attention mechanism for sentence embedding. We design DSA by modifying dynamic routing in capsule network (Sabouretal.,2017) for natural language processing. DSA attends to informative words with a dynamic weight vector. We achieve new state-of-the-art results among sentence encoding methods in Stanford Natural Language Inference (SNLI) dataset with the least number of parameters, while showing comparative results in Stanford Sentiment Treebank (SST) dataset.
CERN Project Sees Orders-of-Magnitude Speedup with AI Approach
An award-winning effort at CERN has demonstrated potential to significantly change how the physics based modeling and simulation communities view machine learning. The CERN team demonstrated that AI-based models have the potential to act as orders-of-magnitude-faster replacements for computationally expensive tasks in simulation, while maintaining a remarkable level of accuracy. Dr. Federico Carminati (Project Coordinator, CERN) points out, "This work demonstrates the potential of'black box' machine-learning models in physics-based simulations." A poster describing this work was awarded the prize for best poster in the category'programming models and systems software' at ISC'18. This recognizes the importance of the work, which was carried out by Dr. Federico Carminati, Gul Rukh Khattak, and Dr. Sofia Vallecorsa at CERN, as well as Jean-Roch Vlimant at Caltech.
China Is Leading in Artificial Intelligence--and American Businesses Should Take Note
Getting a foothold in the Chinese market is a dream for any business. But keep in mind that, in the age of A.I., businesses that can use machine and deep learning techniques to mine, refine, and make products from data culled from all areas of operation--from customer service to employee productivity--will gain a big edge. And access to the Chinese market for U.S. startups comes at a steep price: their business data. With that, and the data from China's almost 1.4 billion citizens, the BAT could soon command what's perhaps our most valuable resource--human data--without the privacy and security restrictions common in much of the rest of the world. In other words: If data is the new oil, China is the new OPEC. It will soon wield tremendous influence in global digital commerce, autonomous vehicles, and a renewed race to outer space.
The Future Of Artificial Intelligence In Health Care
Development in Artificial Intelligence (AI) is unending. Everyday breakthroughs are achieved to further solve problems. From indications, this technology now seems to be veering towards healthcare. Tech developers have indicated that AI is the solution for all problems. DeepMind Technologies Limited, a tech company owned by Google's parent company, is researching into the possibility of using AI in health care, specifically in diagnosing eye conditions.
Schooling Flappy Bird: A Reinforcement Learning Tutorial
In classical programming, software instructions are explicitly made by programmers and nothing is learned from the data at all. In contrast, machine learning is a field of computer science which uses statistical methods to enable computers to learn and to extract knowledge from the data without being explicitly programmed. In this reinforcement learning tutorial, I'll show how we can use PyTorch to teach a reinforcement learning neural network how to play Flappy Bird. But first, we'll need to cover a number of building blocks. Machine learning algorithms can roughly be divided into two parts: Traditional learning algorithms and deep learning algorithms. Traditional learning algorithms usually have much fewer learnable parameters than deep learning algorithms and have much less learning capacity. Also, traditional learning algorithms are not able to do feature extraction: Artificial intelligence specialists need to figure out a good data representation which is then sent to the learning algorithm. Examples of traditional machine learning techniques include SVM, random forest, decision tree, and $k$-means, whereas the central algorithm in deep learning is the deep neural network. The input to a deep neural network can be raw images, and an artificial intelligence specialist doesn't need to find any data representation--the neural network finds the best representation during the training process. A lot of deep learning techniques have been known for a very long time, but recent advances in hardware rapidly boosted deep learning research and development.
Nvidia GeForce RTX: Every game that supports real-time ray tracing and Deep Learning Super Sampling
Nvidia revealed the boundary-pushing GeForce RTX 20-series on Monday, unleashing GeForce RTX 2070, RTX 2080, and RTX 2080 Ti graphics cards brimming with fancy new tech that promises to support fancy new gaming capabilities. Foremost among those feats is real-time ray tracing, the ultra-difficult realistic lighting technology that gives Nvidia's new cards their "RTX" moniker. The RTX cards also support Deep Learning Super-Sampling (DLSS), a fresh Nvidia super-sampling method that puts the AI tensors cores embedded within the GPUs to work. Now, we know which PC games will support them--a crucial step, since all the luxurious tech in the world means nothing if games don't actually tap into it. Both real-time ray tracing and DLSS will debut with a solid backing, as made clear by Nvidia's games partner announcement.