Deep Learning
Deep Learning Advances from IBM Research Part of Watson Studio
Today, with contributions made by IBM scientists, IBM introduces Deep Learning as a Service within Watson Studio, a rich set of cloud-based tools for developers and data scientists to help remove the barriers of training deep learning models in the enterprise. Deep learning and machine learning require expensive hardware and software resources as well as more expensive skilled scientists and developers. Deep learning, in particular, requires users to be experts at different levels of the stack, from neural network design to new hardware. Allowing them to be more effective requires cross-stack innovation and software/hardware co-design. The challenges faced in creating AI models and applications have recently been gaining more attention, with Berkeley's Joe Hellerstein highlighting the AI Engineering Gap, a new academic conference (SysML) focused solely on the intersection of systems and machine learning, and the Stanford DAWN initiative calling out the "lack of systems and tools for end to end machine learning development."
5 Actual Big Data Uses That Make Our Life Better
The Big Data Revolution (with the help of Machine Learning, Deep Learning and Artificial Intelligence) can transform our world in a "Big Brother Nightmare"… It could be… but, The Big Data can at the opposite improve our lives for a better tomorrow. So, let's check these 5 actual cases where the Big Data ecosystem improve humanity lives for real. Data analysis, Deep Learning and robotics improve patient follow-up, long-term care and help prevent relapses. Big Data is used to predict which patients are likely to follow their doctor's advice, and who may not. Mobile applications are developed to check whether a patient is taking medication, such as an inhaler with a GPS chip for asthmatics.
How Alibaba Used Reinforcement Learning To Change Real-Time Bidding
Bidding optimisation is considered among toughest critical problems in online advertising. Bidding strategies adopt different search pattern, for example, Sponsored Search (SS) which depends on the randomness of the user's behaviour and the nature of the platform. Display advertising is considered as one of the simple techniques for auction and has taken over Real-Time Bidding resulting in a better performance for the advertisers. In this article, we will explore how Deep Learning techniques are implemented to optimise the Sponsored Search Real Time Bidding (SS-RTB) system in a stochastic environment. A Reinforcement Learning solution for handling the stochastic environment is proposed in the paper titled Deep Reinforcement Learning For Sponsored Search Real Time Bidding by Alibaba group, where the state transition probability is considered for every two days.
Deep, really Deep Learning
You have heard of a thing called the Internet. Not really sure what it does, but apparently it is going to be really cool and will get really big with time. I choose that year, because that is when I first connected to the internet ( using Compuserve and a VERY slow dial-up modem). In any case, since you are reading this post, you know now, that things have gone far beyond anything we could have imagined back then in 1994. We could really say that things have grown Exponentially.
Difference between AI, Machine Learning and Deep Learning
As we reached the digital era, where computers became an integral part of the everyday lifestyle, people cannot help but be amazed at how far we have come since the time immemorial. The creation of the computers, as well as the internet, had led us into a more complex thinking, making information available to us with just a click. You just type in the words and information will be readily available for you. However, as we approached this era, a lot of inventions and terms became confusing. Have you heard about Artificial intelligence?
Robots Learn by Watching Human Behavior NVIDIA Blog
Robots following coded instructions to complete a task? Robots learning to do things by watching how humans do it? Stanford's Animesh Garg and Marynel Vázquez shared their research in a talk on "Generalizable Autonomy for Robotic Mobility and Manipulation" at the GPU Technology Conference last week. In lay terms, generalizable autonomy is the idea that a robot can observe human behavior, and learn to imitate it in a way that's applicable to a variety of tasks and situations. Learning to cook by watching YouTube videos, for one.
Google's DeepMind opens new AI lab in Paris - SiliconANGLE
DeepMind Technologies Inc., the machine learning company owned by Alphabet Inc., announced today that it's opening a new artificial intelligence lab in Paris. The new lab will be headed by Remi Munos (pictured), a French native and senior researcher at DeepMind who has authored 150 research papers. In an announcement video, Munos said Paris is a perfect fit for DeepMind's next lab because the city has a thriving AI and machine learning ecosystem that's still growing. "Effectively, there are a large number of research labs in universities, engineering schools and public research centers together with a large number of AI startups who have appeared, as well as large companies that are setting themselves up," said Munos. "Joining this network is a very positive move for DeepMind, to collaborate with this scientific community in order to contribute to research and also to teach students." Frédérique Vidal, France's Minister of Higher Education, Research and Innovation, said in a statement that DeepMind's Paris lab "demonstrates the excellence and attractiveness of the Artificial Intelligence Ecosystem in France," and she added that the country will soon establish partnerships with "the public actors of French research."
GoSGD: Distributed Optimization for Deep Learning with Gossip Exchange
Blot, Michael, Picard, David, Cord, Matthieu
With deep convolutional neural networks (CNN) introduced by [1] and [2], computer vision tasks and more specifically image classification have made huge improvements in the years following [3]. CNN performances benefit a lot from big collections of annotated images like [4] or [5]. They are trained by optimizing a loss function with gradient descents computed on random mini-batches according to [6]. The method called stochastic gradient descent (SGD) has proved to be very efficient to train neural networks in general. However current CNN structures are extremely deep like the 100 layers ResNet of [7] and contains a lot of parameters (around 60M for Alexnet [3] and 130M for vgg [8]). Those structures involve heavy gradient computation times making the training on big data-sets very slow. Computation on GPU accelerates the training but requires huge local memory caches. Nevertheless the mini-batch optimization seems suitable for distributing the training over several threads. Many methods have been proposed like 1 [9, 10], which propose to distribute the batches over different threads called workers that periodically exchange information via a central thread to synchronize their models.
Review of Deep Learning
Zhang, Rong, Li, Weiping, Mo, Tong
In recent years, China, the United States and other countries, Google and other high-tech companies have increased investment in artificial intelligence. Deep learning is one of the current artificial intelligence research's key areas. This paper analyzes and summarizes the latest progress and future research directions of deep learning. Firstly, three basic models of deep learning are outlined, including multilayer perceptrons, convolutional neural networks, and recurrent neural networks. On this basis, we further analyze the emerging new models of convolution neural networks and recurrent neural networks. This paper then summarizes deep learning's applications in many areas of artificial intelligence, including voice, computer vision, natural language processing and so on. Finally, this paper discusses the existing problems of deep learning and gives the corresponding possible solutions.
Boosting Handwriting Text Recognition in Small Databases with Transfer Learning
Aradillas, José Carlos, Murillo-Fuentes, Juan José, Olmos, Pablo M.
In this paper we deal with the offline handwriting text recognition (HTR) problem with reduced training datasets. Recent HTR solutions based on artificial neural networks exhibit remarkable solutions in referenced databases. These deep learning neural networks are composed of both convolutional (CNN) and long short-term memory recurrent units (LSTM). In addition, connectionist temporal classification (CTC) is the key to avoid segmentation at character level, greatly facilitating the labeling task. One of the main drawbacks of the CNNLSTM-CTC (CLC) solutions is that they need a considerable part of the text to be transcribed for every type of calligraphy, typically in the order of a few thousands of lines. Furthermore, in some scenarios the text to transcribe is not that long, e.g. in the Washington database. The CLC typically overfits for this reduced number of training samples. Our proposal is based on the transfer learning (TL) from the parameters learned with a bigger database. We first investigate, for a reduced and fixed number of training samples, 350 lines, how the learning from a large database, the IAM, can be transferred to the learning of the CLC of a reduced database, Washington. We focus on which layers of the network could be not re-trained. We conclude that the best solution is to re-train the whole CLC parameters initialized to the values obtained after the training of the CLC from the larger database. We also investigate results when the training size is further reduced. The differences in the CER are more remarkable when training with just 350 lines, a CER of 3.3% is achieved with TL while we have a CER of 18.2% when training from scratch. As a byproduct, the learning times are quite reduced. Similar good results are obtained from the Parzival database when trained with this reduced number of lines and this new approach.