Data (and Computer) Scientists have been working for a long time on improving the ability for algorithms to derive meaning from natural (human) languages -- whether they're trying to create a bot that responds to users questions on their website or determine whether people love or hate their brand on Twitter. The bad news is there is still a deep stack of concepts that you need to understand to tune your results. The good news is that with tools like BERT and ERNIE, getting good results from Natural Language Processing (NLP) is more accessible than ever -- even with modestly sized data sets and computing budgets. Plus, who wouldn't want to do NLP with the Sesame Street crew?! Let's start with a brief look at the history of the discipline. It's possible to break down the development of NLP systems into three broad phases: Deep learning has transformed the practice of NLP over the last decade.

I was at Google DeepMind working on Robotics and Vision. My other interest include NLP and games. We provide a compression program that is stronger than zip, 7z, and tar.gz, and show how it can be used to make intelligent decisions. In particular, we use it to reconstruct the phylogeny tree of mammals and characterize the SARS virus.

The rapid adoption of AI technologies shows that they cater to the needs of businesses. Today, Artificial Intelligence and Machine Learning are seen as part of the everyday life of large organizations in various fields. The rapid pace of AI development – achievements in unmanned aerial vehicles, the ability to beat humans at chess and poker, automated customer service and analytical systems – shows that AI is a revolutionary technology designed to change the way people use devices and conduct business. The appearance of intelligent machines capable of solving difficult problems in a way that only humans could previously hasn't gone unnoticed. From the day of its invention until now, the AI technology has been evolving and improving.

A study co-written by a Southwest Research Institute scientist describes a new algorithm that combines the capabilities of two spacecraft instruments, which could result in lower cost and higher efficiency space missions. The virtual "super instrument," is a computer algorithm that utilizes deep learning to analyze ultraviolet images of the Sun, taken by NASA's Solar Dynamics Observatory, and measure the energy that the Sun emits as ultraviolet light. "Deep learning is an emerging capability that is revolutionizing the way we interact with data," said Dr. Andrés Muñoz-Jaramillo, senior research scientist at SwRI. Muñoz-Jaramillo co-authored the study, published this month in Science Advances, alongside collaborators from nine other institutions as part of NASA's Frontier Development Laboratory. The laboratory is an applied artificial intelligence research accelerator that applies deep learning and machine learning techniques to challenges in space science and exploration.

Paperspace has always had a firm focus on data science teams building machine models, offering them access to GPUs in the cloud, but the company has had broader ambition beyond providing pure infrastructure, and today it announced a new set of tools to help these teams pass the model off to developers and operations in a smoother way in a multi-cloud or hybrid environment. Co-founder and CEO Dillon Erb says this an attempt to provide a full tool set for data scientists and developers, beyond providing pure GPU power to test and build the models. "Machine learning teams do a lot of GPU work -- and as you know, we've been working with GPUs for a number of years now, and that's one of our specialties. Now what we're doing is taking a kind of agile methodology approach or CI/CD (continuous integration/continuous delivery) for machine learning, and using that to solve much larger scale [machine learning] problems," Erb said. As the company describes it, "The new release introduces GradientCI, the industry's first comprehensive CI/CD engine for building, training and deploying deep learning models…" Erb says the goal is to provide a way to take the model built on top of Paperspace and put it to work in the company faster.

DeepMind, a subsidiary of Alphabet (Google's parent company) is working to remove the inherent human biases from machine learning algorithms. The increased deployment of artificial intelligence and machine learning algorithms into the real world has coincided with increased concerns over biases in the algorithms' decision making. From loan and job applications to surveillance and even criminal justice, AI has been shown to exhibit bias – particularly in terms of race and gender – in its decision making. Researchers at DeepMind believe they've developed a useful framework for identifying and removing unfairness in AI decision making. Called Causal Bayesian Networks (CBNs), these are visual representations of datasets that can identify causal relationships within the data and help experts identify factors that might be unfairly weighed against or skewing others.

Sponsored Artificial intelligence and machine learning hold out the promise of enabling businesses to work smarter and faster, by improving and streamlining operations or offering firms the chance to gain a competitive advantage over their rivals. But where is best to host such applications – in the cloud, or locally, at the edge? Despite all the hype, it is early days for the technologies that we loosely label "AI", and many organisations lack the expertise and resources to really take advantage of it. Machine learning and deep learning often require teams of experts, for example, as well as access to large data sets for training, and specialised infrastructure with a considerable amount of processing power. This is because cloud service providers have a wealth of development tools and other resources readily available such as pre-trained deep neural networks for voice, text, image, and translation processing, according to Moor Insights & Strategy Senior Analyst Karl Freund.

The importance of machine learning and deep learning is no longer in doubt. After decades of promise, hype, and disappointment, both have led to practical applications. We haven't gotten to the point where machine learning or deep learning applications are perfect, but many are very good indeed. Of all the excellent machine learning and deep learning frameworks available, TensorFlow is the most mature, has the most citations in research papers (even excluding citations from Google employees), and has the best story about use in production. It may not be the easiest framework to learn, but it's much less intimidating than it was in 2016.

Based on Google's BERT pre-training model, RoBERTa includes additional pre-training improvements that achieve state-of-the-art results on several benchmarks, using only unlabeled text from the world-wide web, with minimal fine-tuning and no data augmentation. The Facebook team announced their work in a recent blog post as "part of Facebook's ongoing commitment to advancing the state-of-the-art in self-supervised systems that can be developed with less reliance on time- and resource-intensive data labeling." The team re-implemented Google's BERT neural-network architecture in PyTorch, made several changes to the model's hyperparameters, and trained the network with an order-of-magnitude more data and for more iterations. The model was evaluated on three common NLP benchmarks: General Language Understanding Evaluation (GLUE), Stanford Question Answering Dataset (SQuAD), and ReAding Comprehension from Examinations (RACE). RoBERTa outperformed BERT on these tests, and in some cases also outperformed the current leading model, XLNet.

Dee learning is getting a lot of hype at the moment. People want to use neural networks everywhere, but are they always the right choice? We'll take a look at some of the disadvantages of using them. There are four primary reasons why deep learning enjoys so much buzz at the moment: data, computational power, the algorithm itself and marketing. Massive amounts of available data gathered over the last decade has contributed greatly to the popularity of deep learning. This has allowed neural networks to really show their potential since they get better the more data you fed into them.