Deep Learning
Google's TensorFlow is ready for quantum, but is AI ready for quantum?
Spoiler alert: Quantum computers may not make your cats and dogs classifiers go any faster. Here's how you can still get a free Windows 10 upgrade You can still use Microsoft's free upgrade tools to install Windows 10 on an old PC running Windows 7 or Windows 8.1. No product key is required, and the digital license says you're activated and ready to go. Google this week announced a new version of its TensorFlow framework for building machine learning models, a kind of mash-up between TensorFlow and Cinq, another framework developed at Google that's designed for building quantum computing algorithms. Together, they could let you build a deep learning model to run on a future quantum computer with no more than a bunch of lines of Python.
Reinforcement Learning: The Algorithms Changing How Computers Make Decisions
The last decade of tech was to a large part defined by the advent of Deep Supervised Learning (DL). The availability of cheap data at scale, computational power, and researcher interest have made it the de-facto school of algorithms used for most pattern recognition problems. Face recognition on social media, product recommendations on sites, voice assistants like Google Assistant, Alexa, and Siri are some examples largely powered by DL. The issue with deep learning is that the resources that led to its rise are also giving rise to inequities. Today, it is tough for startups to beat'big tech' like Apple, Google, Amazon, and Microsoft in deep learning through better research capabilities or better data.
RPI targets the coronavirus with deep-learning, artificial intelligence supercomputer
Rensselaer Polytechnic Institute is turning one of the most powerful supercomputers in the world against COVID-19. AiMOS, short for Artificial Intelligence Multiprocessing Optimized System, can perform eight quadrillion calculations per second and is uniquely capable of exploring new applications in artificial intelligence. RPI is offering the supercomputer to the research community--including government entities, academic universities, national labs, and private businesses--to support of new coronavirus research. To combat the devastating effects of this pandemic, we must be able to fully grasp the complexities and interconnectedness of biological systems and epidemiological data, as researchers work to develop therapeutic interventions and address gaps in our knowledge. AiMOS is the most powerful supercomputer at a private university, the third-most energy-efficient supercomputer in the world, and the 24th most-powerful in the world overall.
EikoNet: Solving the Eikonal equation with Deep Neural Networks
Smith, Jonathan D., Azizzadenesheli, Kamyar, Ross, Zachary E.
The recent deep learning revolution has created an enormous opportunity for accelerating compute capabilities in the context of physics-based simulations. Here, we propose EikoNet, a deep learning approach to solving the Eikonal equation, which characterizes the first-arrival-time field in heterogeneous 3D velocity structures. Our grid-free approach allows for rapid determination of the travel time between any two points within a continuous 3D domain. These travel time solutions are allowed to violate the differential equation - which casts the problem as one of optimization - with the goal of finding network parameters that minimize the degree to which the equation is violated. In doing so, the method exploits the differentiability of neural networks to calculate the spatial gradients analytically, meaning the network can be trained on its own without ever needing solutions from a finite difference algorithm. EikoNet is rigorously tested on several velocity models and sampling methods to demonstrate robustness and versatility. Training and inference are highly parallelized, making the approach well-suited for GPUs. EikoNet has low memory overhead, and further avoids the need for travel-time lookup tables. The developed approach has important applications to earthquake hypocenter inversion, ray multi-pathing, and tomographic modeling, as well as to other fields beyond seismology where ray tracing is essential.
Multi-Lead ECG Classification via an Information-Based Attention Convolutional Neural Network
Tung, Hao, Zheng, Chao, Mao, Xinsheng, Qian, Dahong
Objective: A novel structure based on channel-wise attention mechanism is presented in this paper. Embedding with the proposed structure, an efficient classification model that accepts multi-lead electrocardiogram (ECG) as input is constructed. Methods: One-dimensional convolutional neural networks (CNN) have proven to be effective in pervasive classification tasks, enabling the automatic extraction of features while classifying targets. We implement the Residual connection and design a structure which can learn the weights from the information contained in different channels in the input feature map during the training process. An indicator named mean square deviation is introduced to monitor the performance of a particular model segment in the classification task on the two out of the five ECG classes. The data in the MIT-BIH arrhythmia database is used and a series of control experiments is conducted. Results: Utilizing both leads of the ECG signals as input to the neural network classifier can achieve better classification results than those from using single channel inputs in different application scenarios. Models embedded with the channel-wise attention structure always achieve better scores on sensitivity and precision than the plain Resnet models. The proposed model exceeds the performance of most of the state-of-the-art models in ventricular ectopic beats (VEB) classification, and achieves competitive scores for supraventricular ectopic beats (SVEB). Conclusion: Adopting more lead ECG signals as input can increase the dimensions of the input feature maps, helping to improve both the performance and generalization of the network model. Significance: Due to its end-to-end characteristics, and the extensible intrinsic for multi-lead heart diseases diagnosing, the proposed model can be used for the real-time ECG tracking of ECG waveforms for Holter or wearable devices.
Model-based Asynchronous Hyperparameter Optimization
Tiao, Louis C., Klein, Aaron, Archambeau, Cedric, Seeger, Matthias
We introduce a model-based asynchronous multi-fidelity hyperparameter optimization (HPO) method, combining strengths of asynchronous Hyperband and Gaussian process-based Bayesian optimization. Our method obtains substantial speed-ups in wall-clock time over, both, synchronous and asynchronous Hyperband, as well as a prior model-based extension of the former. Candidate hyperparameters to evaluate are selected by a novel jointly dependent Gaussian process-based surrogate model over all resource levels, allowing evaluations at one level to be informed by evaluations gathered at all others. We benchmark several covariance functions and conduct extensive experiments on hyperparameter tuning for multi-layer perceptrons on tabular data, convolutional networks on image classification, and recurrent networks on language modelling, demonstrating the benefits of our approach.
Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods
Zhi, Jiale, Wang, Rui, Clune, Jeff, Stanley, Kenneth O.
Recent advances in machine learning are consistently enabled by increasing amounts of computation. Reinforcement learning (RL) and population-based methods in particular pose unique challenges for efficiency and flexibility to the underlying distributed computing frameworks. These challenges include frequent interaction with simulations, the need for dynamic scaling, and the need for a user interface with low adoption cost and consistency across different backends. In this paper we address these challenges while still retaining development efficiency and flexibility for both research and practical applications by introducing Fiber, a scalable distributed computing framework for RL and population-based methods. Fiber aims to significantly expand the accessibility of large-scale parallel computation to users of otherwise complicated RL and population-based approaches without the need to for specialized computational expertise.
Integrating Physiological Time Series and Clinical Notes with Deep Learning for Improved ICU Mortality Prediction
Shukla, Satya Narayan, Marlin, Benjamin M.
Intensive Care Unit Electronic Health Records (ICU EHRs) store multimodal data about patients including clinical notes, sparse and irregularly sampled physiological time series, lab results, and more. To date, most methods designed to learn predictive models from ICU EHR data have focused on a single modality. In this paper, we leverage the recently proposed interpolation-prediction deep learning architecture(Shukla and Marlin 2019) as a basis for exploring how physiological time series data and clinical notes can be integrated into a unified mortality prediction model. We study both early and late fusion approaches and demonstrate how the relative predictive value of clinical text and physiological data change over time. Our results show that a late fusion approach can provide a statistically significant improvement in mortality prediction performance over using individual modalities in isolation.
Capturing and Explaining Trajectory Singularities using Composite Signal Neural Networks
Dubois, Hippolyte, Callet, Patrick Le, Coutrot, Antoine
Spatial trajectories are ubiquitous and complex signals. Their analysis is crucial in many research fields, from urban planning to neuroscience. Several approaches have been proposed to cluster trajectories. They rely on hand-crafted features, which struggle to capture the spatio-temporal complexity of the signal, or on Artificial Neural Networks (ANNs) which can be more efficient but less interpretable. In this paper we present a novel ANN architecture designed to capture the spatio-temporal patterns characteristic of a set of trajectories, while taking into account the demographics of the navigators. Hence, our model extracts markers linked to both behaviour and demographics. We propose a composite signal analyser (CompSNN) combining three simple ANN modules. Each of these modules uses different signal representations of the trajectory while remaining interpretable. Our CompSNN performs significantly better than its modules taken in isolation and allows to visualise which parts of the signal were most useful to discriminate the trajectories.
Dividing Deep Learning Model for Continuous Anomaly Detection of Inconsistent ICT Systems
Tajiri, Kengo, Ikeda, Yasuhiro, Nakano, Yuusuke, Watanabe, Keishiro
Health monitoring is important for maintaining reliable information and communications technology (ICT) systems. Anomaly detection methods based on machine learning, which train a model for describing "normality" are promising for monitoring the state of ICT systems. However, these methods cannot be used when the type of monitored log data changes from that of training data due to the replacement of certain equipment. Therefore, such methods may dismiss an anomaly that appears when log data changes. To solve this problem, we propose an ICT-systems-monitoring method with deep learning models divided based on the correlation of log data. We also propose an algorithm for extracting the correlations of log data from a deep learning model and separating log data based on the correlation. When some of the log data changes, our method can continue health monitoring with the divided models which are not affected by changes in the log data. We present the results from experiments involving benchmark data and real log data, which indicate that our method using divided models does not decrease anomaly detection accuracy and a model for anomaly detection can be divided to continue monitoring a network state even if some the log data change.