Deep Learning
MTSS: Learn from Multiple Domain Teachers and Become a Multi-domain Dialogue Expert
Peng, Shuke, Ji, Feng, Lin, Zehao, Cui, Shaobo, Chen, Haiqing, Zhang, Yin
How to build a high-quality multi-domain dialogue system is a challenging work due to its complicated and entangled dialogue state space among each domain, which seriously limits the quality of dialogue policy, and further affects the generated response. In this paper, we propose a novel method to acquire a satisfying policy and subtly circumvent the knotty dialogue state representation problem in the multi-domain setting. Inspired by real school teaching scenarios, our method is composed of multiple domain-specific teachers and a universal student. Each individual teacher only focuses on one specific domain and learns its corresponding domain knowledge and dialogue policy based on a precisely extracted single domain dialogue state representation. Then, these domain-specific teachers impart their domain knowledge and policies to a universal student model and collectively make this student model a multi-domain dialogue expert. Experiment results show that our method reaches competitive results with SOTAs in both multi-domain and single domain setting.
Automated Question Answer medical model based on Deep Learning Technology
Abdallah, Abdelrahman, Kasem, Mahmoud, Hamada, Mohamed, Sdeek, Shaymaa
Artificial intelligence can now provide more solutions for different problems, especially in the medical field. One of those problems the lack of answers to any given medical/health-related question. The Internet is full of forums that allow people to ask some specific questions and get great answers for them. Nevertheless, browsing these questions in order to locate one similar to your own, also finding a satisfactory answer is a difficult and time-consuming task. This research will introduce a solution to this problem by automating the process of generating qualified answers to these questions and creating a kind of digital doctor. Furthermore, this research will train an end-to-end model using the framework of RNN and the encoder-decoder to generate sensible and useful answers to a small set of medical/health issues. The proposed model was trained and evaluated using data from various online services, such as WebMD, HealthTap, eHealthForums, and iCliniq.
Learning to Recommend Signal Plans under Incidents with Real-Time Traffic Prediction
The main question to address in this paper is to recommend optimal signal timing plans in real time under incidents by incorporating domain knowledge developed with the traffic signal timing plans tuned for possible incidents, and learning from historical data of both traffic and implemented signals timing. The effectiveness of traffic incident management is often limited by the late response time and excessive workload of traffic operators. This paper proposes a novel decision-making framework that learns from both data and domain knowledge to real-time recommend contingency signal plans that accommodate non-recurrent traffic, with the outputs from real-time traffic prediction at least 30 minutes in advance. Specifically, considering the rare occurrences of engagement of contingency signal plans for incidents, we propose to decompose the end-to-end recommendation task into two hierarchical models: real-time traffic prediction and plan association. We learn the connections between the two models through metric learning, which reinforces partial-order preferences observed from historical signal engagement records. We demonstrate the effectiveness of our approach by testing this framework on the traffic network in Cranberry Township in 2019. Results show that our recommendation system has a precision score of 96.75% and recall of 87.5% on the testing plan, and make recommendation of an average of 22.5 minutes lead time ahead of Waze alerts. The results suggest that our framework is capable of giving traffic operators a significant time window to access the conditions and respond appropriately.
Inverse Estimation of Elastic Modulus Using Physics-Informed Generative Adversarial Networks
Warner, James E., Cuevas, Julian, Bomarito, Geoffrey F., Leser, Patrick E., Leser, William P.
While standard generative adversarial networks (GANs) rely solely on training data to learn unknown probability distributions, physics-informed GANs (PI-GANs) encode physical laws in the form of stochastic partial differential equations (PDEs) using auto differentiation. By relating observed data to unobserved quantities of interest through PDEs, PI-GANs allow for the estimation of underlying probability distributions without their direct measurement (i.e. inverse problems). The scalable nature of GANs allows high-dimensional, spatially-dependent probability distributions (i.e., random fields) to be inferred, while incorporating prior information through PDEs allows the training datasets to be relatively small. In this work, PI-GANs are demonstrated for the application of elastic modulus estimation in mechanical testing. Given measured deformation data, the underlying probability distribution of spatially-varying elastic modulus (stiffness) is learned. Two feed-forward deep neural network generators are used to model the deformation and material stiffness across a two dimensional domain. Wasserstein GANs with gradient penalty are employed for enhanced stability. In the absence of explicit training data, it is demonstrated that the PI-GAN learns to generate realistic, physically-admissible realizations of material stiffness by incorporating the PDE that relates it to the measured deformation. It is shown that the statistics (mean, standard deviation, point-wise distributions, correlation length) of these generated stiffness samples have good agreement with the true distribution.
TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids
Fedorov, Igor, Stamenovic, Marko, Jensen, Carl, Yang, Li-Chia, Mandell, Ari, Gan, Yiming, Mattina, Matthew, Whatmough, Paul N.
Modern speech enhancement algorithms achieve remarkable noise suppression by means of large recurrent neural networks (RNNs). However, large RNNs limit practical deployment in hearing aid hardware (HW) form-factors, which are battery powered and run on resource-constrained microcontroller units (MCUs) with limited memory capacity and compute capability. In this work, we use model compression techniques to bridge this gap. We define the constraints imposed on the RNN by the HW and describe a method to satisfy them. Although model compression techniques are an active area of research, we are the first to demonstrate their efficacy for RNN speech enhancement, using pruning and integer quantization of weights/activations. We also demonstrate state update skipping, which reduces the computational load. Finally, we conduct a perceptual evaluation of the compressed models to verify audio quality on human raters. Results show a reduction in model size and operations of 11.9$\times$ and 2.9$\times$, respectively, over the baseline for compressed models, without a statistical difference in listening preference and only exhibiting a loss of 0.55dB SDR. Our model achieves a computational latency of 2.39ms, well within the 10ms target and 351$\times$ better than previous work.
Neural Architecture Search for Gliomas Segmentation on Multimodal Magnetic Resonance Imaging
Past few years have witnessed the artificial intelligence inspired evolution in various medical fields. The diagnosis and treatment of gliomas -- one of the most commonly seen brain tumors with low survival rate -- rely heavily on the computer assisted segmentation process undertaken on the magnetic resonance imaging (MRI) scans. Although the encoder-decoder shaped deep learning networks have been the de facto standard style for semantic segmentation tasks in medical imaging analysis, enormous effort is still required to be spent on designing the detailed architecture of the down-sampling and up-sampling blocks. In this work, we propose a neural architecture search (NAS) based solution to brain tumor segmentation tasks on multimodal volumetric MRI scans. Three sets of candidate operations are composed respectively for three kinds of basic building blocks in which each operation is assigned with a specific probabilistic parameter to be learned. Through alternately updating the weights of operations and the other parameters in the network, the searching mechanism ends up with two optimal structures for the upward and downward blocks. Moreover, the developed solution also integrates normalization and patching strategies tailored for brain MRI processing. Extensive comparative experiments on the BraTS 2019 dataset demonstrate that the proposed algorithm not only could relieve the pressure of fabricating block architectures but also possesses competitive feasibility and scalability.
Chinese Tech & Utilities Best Ideas for International Investors
It is hard right now to think about how emerging markets and international companies are dealing with the slowdown, especially when you think of how integrated global supply chains are, and whether that is going to continue. Thankfully with the help of our Artificial Intelligence ("AI") deep learning algorithms, that study fundamental and price data in addition to alternative data like article sentiment and social sentiment, we are able to narrow down the top buys and top shorts in international ETFs. Going global can help with overall diversification if you know the smart moves to make. One of the top buys as rated by our AI system is the Invesco China Technology ETF (CQQQ). Investors agree that this is an interesting play, as China is well on its way to be a global powerhouse in technology, politics aside.
What is deep learning?
Most of the time when we are talking about AI, we are thinking about a very specific form of AI called machine learning. Machine learning systems use a range of mathematical processes to learn procedures and perform tasks automatically. You don't need to exactly mimic human intelligence in order to create a system that appears intelligent. Instead, you can model a range of problem solving architectures that are comparatively simple to understand. One type of machine learning approach that is very important at the moment is deep learning.
Microsoft teamed up with OpenAI to build a massive AI supercomputer in Azure โ TechCrunch
At its Build developer conference, Microsoft today announced that it has teamed up with OpenAI, the startup trying to build a general artificial intelligence, with -- among other things -- a $1 billion investment from Microsoft, to create one of the world's fastest supercomputers on top of Azure's infrastructure. Microsoft says that the 285,000-core machine would have ranked in the top five of the TOP500 supercomputer rankings. Because Microsoft doesn't actually tell us much more than that, except for a few more specs that say it had 10,000 GPUs and 400 gigabits per second of network connectivity per server, we'll just have to take Microsoft's and OpenAI's word for this. To be in the top five of supercomputers, a machine would currently have to reach more than 23,000 teraflops per second. It's also worth noting that the No. 1 machine, the IBM Power System-based Summit, reaches over 148,000 teraflops, so there is quite a wide margin here.
Accelerating Medical Image Segmentation with NVIDIA Tensor Cores and TensorFlow 2 NVIDIA Developer Blog
Medical image segmentation is a hot topic in the deep learning community. Proof of that is the number of challenges, competitions, and research projects being conducted in this area, which only rises year over year. Among all the different approaches to this problem, U-Net has become the backbone of many of the top-performing solutions for both 2D and 3D segmentation tasks. This is due to its simplicity, versatility, and effectiveness. When practitioners are confronted with a new segmentation task, the first step commonly is to use an existent implementation of U-Net as a backbone.