Deep Learning
Time Series Forecasting With Deep Learning: A Survey
While traditional methods have focused on parametric models informed by domain expertise - such as autoregressive (AR) [6], exponential smoothing [7, 8] or structural time series models [9] - modern machine learning methods provide a means to learn temporal dynamics in a purely data-driven manner [10]. With the increasing data availability and computing power in recent times, machine learning has become a vital part of the next generation of time series forecasting models. Deep learning in particular has gained popularity in recent times, inspired by notable achievements in image classification [11], natural language processing [12] and reinforcement learning [13]. By incorporating bespoke architectural assumptions - or inductive biases [14] - that reflect the nuances of underlying datasets, deep neural networks are able to learn complex data representations [15], which alleviates the need for manual feature engineering and model design. The availability of open-source backpropagation frameworks [16, 17] has also simplified the network training, allowing for the customisation for network components and loss functions.
Measure Utility, Gain Trust: Practical Advice for XAI Researcher
Davis, Brittany, Glenski, Maria, Sealy, William, Arendt, Dustin
Research into the explanation of machine learning models, i.e., explainable AI (XAI), has seen a commensurate exponential growth alongside deep artificial neural networks throughout the past decade. For historical reasons, explanation and trust have been intertwined. However, the focus on trust is too narrow, and has led the research community astray from tried and true empirical methods that produced more defensible scientific knowledge about people and explanations. To address this, we contribute a practical path forward for researchers in the XAI field. We recommend researchers focus on the utility of machine learning explanations instead of trust. We outline five broad use cases where explanations are useful and, for each, we describe pseudo-experiments that rely on objective empirical measurements and falsifiable hypotheses. We believe that this experimental rigor is necessary to contribute to scientific knowledge in the field of XAI.
Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models
Zhang, Lei M., Plappert, Matthias, Zaremba, Wojciech
We propose a method to predict the sim-to-real transfer performance of RL policies. Our transfer metric simplifies the selection of training setups (such as algorithm, hyperparameters, randomizations) and policies in simulation, without the need for extensive and time-consuming real-world rollouts. A probabilistic dynamics model is trained alongside the policy and evaluated on a fixed set of real-world trajectories to obtain the transfer metric. Experiments show that the transfer metric is highly correlated with policy performance in both simulated and real-world robotic environments for complex manipulation tasks. We further show that the transfer metric can predict the effect of training setups on policy transfer performance.
Streaming System Design for Large-Scale Machine Learning Applications
In October of 2019 Crunchbase raised $30M in Series C financing from OMERS Ventures. Crunchbase is charging forward, focusing more deeply on the analysis of business signals for both private and public companies. Here at the Engineering Team, we have been working on the interesting challenge of detecting these high value business signals from various sources, such as Tweets and news articles. Some examples of important signals include funding rounds, acquisitions, and key leadership hires. Finding these signals the moment they are announced empowers our customers to make well-informed business decisions.
Can This Tiny Language Model Defeat Gigantic GPT3?
While GPT-3 has been bragging about achieving state-of-the-art performance on Complex NLP tasks with hundred billion parameters, researchers from the LMU Munich, Germany have proposed a language model who can show similar achievements with way fewer parameters. GPT-3 has been trained on 175 billion parameters and thus showed remarkable few-shot abilities, and by reformulating a few tasks and prompting inputs, it also showed immense capabilities on SuperGLUE benchmark. However it comes with two most significant drawbacks -- large models aren't always feasible for real-world scenarios, and with the context window of these monstrous models is limited to a few hundred tokens, it doesn't scale more than a few examples. And thus, the researchers proposed an alternative to priming, i.e. PET required unlabelled data, which is easier to gather than labelled data, thus making it usable for real-world applications.
Deep learning for next-generation sleep diagnostics
Currently, the diagnosis of sleep disorders relies on polysomnographic recordings with a time-consuming manual analysis with low reliability between different manual scorers. Throughout the night, sleep stages are identified manually in non-overlapping 30-second epochs starting from the onset of the recording based on electroencephalography (EEG), electro-oculography (EOG), and chin electromyography (EMG) signals which require meticulous placement of electrodes. Moreover, the diagnosis of many sleep disorders relies on outdated guidelines. When assessing the severity of obstructive sleep apnea (OSA), the patients are classified based on thresholds of the apnea-hypopnea index (AHI), i.e. the number of respiratory disruptions during sleep. These thresholds are not fully based on solid scientific evidence and remain the same across different measurement techniques.
Global Big Data Conference
How to pick a cloud machine learning platform? To create an effective machine learning and deep learning model, you need more data, a way to clean the data and perform feature engineering on it. It is also a way to train models on your data in a reasonable amount of time. After that, you need a way to install your models, surveil them for drift over time, and retrain them as required. If you have invested in compute resources and accelerators such as GPUs, you can do all of that on-premises.
Global Big Data Conference
For several years, there has been a lot of discussion around AI's capabilities. Many believe that AI will outperform humans in solving certain areas. As the technology is in its infancy, researchers are expecting human-like autonomous systems in the next coming years. OpenAI has a leading stance in the artificial intelligence research space. Founded in December 2015, the company's goal is to advance digital intelligence in a way that can benefit humanity as a whole.
4 Pre-Trained CNN Models to Use for Computer Vision with Transfer Learning
ResNet50 is a convolutional neural network which has a depth of 50 layers. It was build and trained by Microsoft in 2015 and you can access the model performance results on their paper, titled Deep Residual Learning for Image Recognition. This model is also trained on more than 1 million images from the ImageNet database. Just like VGG-19, it can classify up to 1000 objects and the network was trained on 224x224 pixels colored images.