Deep Learning
Artificial intelligence to boost Earth system science
In the past decades mainly static attributes have been investigated using machine learning approaches, such as the distribution of soil properties from the local to the global scale. For some time now, it has been possible to tackle more dynamic processes by using more sophisticated deep learning techniques. This allows for example to quantify the global photosynthesis on land with simultaneous consideration of seasonal and short term variations. "From a plethora of sensors, a deluge of Earth system data has become available, but so far we've been lagging behind in analysis and interpretation," explains Markus Reichstein, managing director of the Max Planck Institute for Biogeochemistry in Jena, directory board member of the Michael-Stifel-Center Jena (MSCJ) and first author of the publication. "This is where deep learning techniques become a promising tool, beyond the classical machine learning applications such as image recognition, natural language processing or AlphaGo," adds co-author Joachim Denzler from the Computer Vision Group of the Friedrich Schiller University Jena (FSU) and member of MSCJ.
Deep Learning GPU Dedicated Servers
For many tasks, such as deep learning (also known as deep structured learning or hierarchical learning), a CPU is no longer enough. In these cases, a GPU will actually help you perform operations significantly faster. This is primarily because with a modern GPU you'll be able to run many more threads, a common CPU may have 16 cores while a common GPU has over 4000 cores. These cores are much simpler and cannot do as much a s CPU, but they don't have to in this case, meaning a GPU allows you to train your neural network much faster. Steadfast GPU dedicated servers can support a wide array of operating systems, such as Windows and most Linux distributions, and thus support most known machine learning and neural network libraries (i.e.
OpenAI's Realistic Text-Generating AI Triggers Ethics Concerns
The release of a text-generating AI has triggered new discussions around the ethics of AI.Andrew Mason, Flickr. OpenAI, the AI research institute founded by Elon Musk, developed an AI which can generate the most realistic paragraphs of text to date. Here's an excerpt generated by the AI when prompted to discuss a fake unicorn discovery: The scientist named the population, after their distinctive horn, Ovid's Unicorn. These four-horned, silver-white unicorns were previously unknown to science. Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
r/MachineLearning - [P] Analyzing Recurrent Neural Networks (RNNs) Using Polymer Dynamics Theory
I'm learning RNN theory and as a project, I tried to better understand the dynamics of LSTM elements when applied to input strings by relating the dynamics to concepts I'm more familiar with in chemical nonequilibrium statistical mechanics. Identified some interesting behavior in terms of the relatively smaller impact of terminal pad characters on the element dynamics versus other characters which cause large changes in the element values. Details in the linked blog post. Assume this behavior is well known, but I wasn't able to find a publication that demonstrates this behavior. Would appreciate learning about prior related work that I should be citing.
Accelerating genomic research with high-performance computing
The vast amount of information encoded in an individual's DNA tells great tales of one's health and disease conditions. When the first human genome was sequenced, the project that began in 1990 took over 10 years and cost around $2.7 billion. According to Andrew Underwood, CTO, HPC & Artificial Intelligence, Dell EMC, Australia and New Zealand, data intensive computing is fast becoming a dominant approach. Especially in R&D, it is a rapidly growing field of research built on data that is generated from scientific instruments, people, machines and IoT devices. Data comes in high velocities and in large volumes โ requiring scientists to harness the power of high performance computing to analyze data faster for timely insights in their field of research.
Semantic Neural Machine Translation using AMR
Song, Linfeng, Gildea, Daniel, Zhang, Yue, Wang, Zhiguo, Su, Jinsong
It is intuitive that semantic representations can be useful for machine translation, mainly because they can help in enforcing meaning preservation and handling data sparsity (many sentences correspond to one meaning) of machine translation models. On the other hand, little work has been done on leveraging semantics for neural machine translation (NMT). In this work, we study the usefulness of AMR (short for abstract meaning representation) on NMT. Experiments on a standard English-to-German dataset show that incorporating AMR as additional knowledge can significantly improve a strong attention-based sequence-to-sequence neural translation model.
A spelling correction model for end-to-end speech recognition
Guo, Jinxi, Sainath, Tara N., Weiss, Ron J.
Attention-based sequence-to-sequence models for speech recognition jointly train an acoustic model, language model (LM), and alignment mechanism using a single neural network and require only parallel audio-text pairs. Thus, the language model component of the end-to-end model is only trained on transcribed audio-text pairs, which leads to performance degradation especially on rare words. While there have been a variety of work that look at incorporating an external LM trained on text-only data into the end-to-end framework, none of them have taken into account the characteristic error distribution made by the model. In this paper, we propose a novel approach to utilizing text-only data, by training a spelling correction (SC) model to explicitly correct those errors. On the LibriSpeech dataset, we demonstrate that the proposed model results in an 18.6% relative improvement in WER over the baseline model when directly correcting top ASR hypothesis, and a 29.0% relative improvement when further rescoring an expanded n-best list using an external LM.
Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
Chen, Changan, Liu, Yuejiang, Kreiss, Sven, Alahi, Alexandre
Mobility in an effective and socially-compliant manner is an essential yet challenging task for robots operating in crowded spaces. Recent works have shown the power of deep reinforcement learning techniques to learn socially cooperative policies. However, their cooperation ability deteriorates as the crowd grows since they typically relax the problem as a one-way Human-Robot interaction problem. In this work, we want to go beyond first-order Human-Robot interaction and more explicitly model Crowd-Robot Interaction (CRI). We propose to (i) rethink pairwise interactions with a self-attention mechanism, and (ii) jointly model Human-Robot as well as Human-Human interactions in the deep reinforcement learning framework. Our model captures the Human-Human interactions occurring in dense crowds that indirectly affects the robot's anticipation capability. Our proposed attentive pooling mechanism learns the collective importance of neighboring humans with respect to their future states. Various experiments demonstrate that our model can anticipate human dynamics and navigate in crowds with time efficiency, outperforming state-of-the-art methods.