Goto

Collaborating Authors

An AI backed by Elon Musk just 'evolved' to learn by itself

#artificialintelligence

Most of today's artificial intelligence (AI) systems rely on machine learning algorithms that can predict specific outcomes by drawing on pre-established values, but now researchers from OpenAI, a company funded by no less than Elon Musk and Peter Thiel, who are trying to democratise AI for "human good" just discovered – literally – that a machine learning system they created to predict the next character in the text of reviews from Amazon evolved into an unsupervised learning system that could learn how to read sentiment. That's a pretty big deal, and it's also something that, at the moment, even the researchers themselves can't explain. "We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment," said OpenAI in a blog. According to the post OpenAI's neural network was able to train itself and analyse sentiment accurately by classifying Amazon's reviews as either positive or negative – and it then generated follow on text that fit with the sentiment. The AI the team used was what's known as a multiplicative long short-term memory (LSTM) model that was trained for a month, processing 12,500 characters a second using Nvidia Pascal GPU's – which Nvidia's own CEO gifted to Elon Musk last year – with "4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text."


An AI backed by Elon Musk just 'evolved' to learn by itself

#artificialintelligence

Most of today's artificial intelligence (AI) systems rely on machine learning algorithms that can predict specific outcomes by drawing on pre-established values, but now researchers from OpenAI, a company funded by no less than Elon Musk and Peter Thiel, who are trying to democratise AI for "human good" just discovered – literally – that a machine learning system they created to predict the next character in the text of reviews from Amazon evolved into an unsupervised learning system that could learn how to read sentiment. That's a pretty big deal, and it's also something that, at the moment, even the researchers themselves can't explain. "We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment," said OpenAI in a blog. According to the post OpenAI's neural network was able to train itself and analyse sentiment accurately by classifying Amazon's reviews as either positive or negative – and it then generated follow on text that fit with the sentiment. The AI the team used was what's known as a multiplicative long short-term memory (LSTM) model that was trained for a month, processing 12,500 characters a second using Nvidia Pascal GPU's – which Nvidia's own CEO gifted to Elon Musk last year – with "4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text."


Deep Learning for NLP, advancements and trends in 2017 - Tryolabs Blog

#artificialintelligence

To create a summary, two different people will use different words and sentence orders, and both summaries will probably be considered as valid. Thus, a good summary does not necessarily have to be a sequence of words that match a sequence in the training dataset as much as possible. Knowing this, the authors avoid the standard teacher forcing algorithm, which minimizes the loss at each decoding step (i.e. for each generated word), and they rely on a reinforcement learning strategy that proves to be an excellent choice.


Deep Dive Into Sentiment Analysis - DZone AI

#artificialintelligence

Sentiment analysis is a gateway to AI-based text analysis. For any company or data scientist looking to extract meaning out of an unstructured text corpus, sentiment analysis is one of the first steps which gives a high RoI of additional insights with relatively low investment of time and effort. With an explosion of text data available in digital formats, the need for sentiment analysis and other NLU techniques for analyzing this data is growing rapidly. Sentiment analysis looks relatively simple and works very well today, but we have reached hereafter significant efforts by researchers who have invented different approaches and tried numerous models. In the chart above, we give a snapshot to the reader about the different approaches tried and their corresponding accuracy on the IMDB dataset.


Semi-supervised Sequence Learning

Neural Information Processing Systems

We present two approaches to use unlabeled data to improve Sequence Learning with recurrent networks. The first approach is to predict what comes next in a sequence, which is a language model in NLP. The second approach is to use a sequence autoencoder, which reads the input sequence into a vector and predicts the input sequence again. These two algorithms can be used as a "pretraining" algorithm for a later supervised sequence learning algorithm. In other words, the parameters obtained from the pretraining step can then be used as a starting point for other supervised training models. In our experiments, we find that long short term memory recurrent networks after pretrained with the two approaches become morestable to train and generalize better. With pretraining, we were able to achieve strong performance in many classification tasks, such as text classification with IMDB, DBpedia or image recognition in CIFAR-10.