NLP may have been a little late to the AI epiphany but it is doing wonders with organizations like Google, OpenAI releasing state-of-the-art(SOTA) language models like BERT and GPT-2/3 respectively. GitHub Copilot and OpenAI codex are among a few very popular applications that are in the news. As someone who has very limited exposure to NLP, I decided to take up NLP as an area of research and the next few blogs/videos will be me sharing what I learn after dissecting some important components of NLP. Top Deep Learning models like BERT, GPT-2, or GPT-3 all share the same components but with different architectures that distinguish one model from another. In this newsletter(and notebook), we are going to focus on the basics of the first component of an NLP pipeline which is tokenization.

OCR or optical character recognition is becoming increasingly important and needed for many companies and organizations that look for ways to automate and streamline their digitalization pipelines. Many companies for example need invoice automation or handwriting recognition. These companies use OCR extensively. With the rise of AI, we can now tackle OCR problems in an unprecedented way. ORPALIS is at the forefront of developing deep learning-based solutions for solving challenging OCR tasks.

With the recent publication on arXiv of the article On the Opportunities and Risks of Foundation Models, this month our experts discuss the concept of such models. Joining the discussion this time are: Sanmay Das (George Mason University), Tom Dietterich (Oregon State University), Stephen Hanson (Rutgers University), Sabine Hauert (University of Bristol), Holger Hoos (Leiden University) and Carles Sierra (Artificial Intelligence Research Institute of the Spanish National Research Council). Sabine Hauert: This month we'll be discussing this paper on the opportunities and risks of foundation models. Could somebody give an explanation of what these models are and some of the issues? Tom Dietterich: So these foundation models, like BERT and GPT-3 are deep neural networks trained in an unsupervised fashion on huge collections of data.

Body composition imaging relies on assessment of tissues composition and distribution. Quantitative data of body composition have been linked to pathogenesis and clinical outcomes of a wide spectrum of diseases, including oncological and cardiovascular. Obesity classification is based on body mass index (BMI) which has the shortcoming not to provide any information on the distribution of adipose tissue and skeletal muscle tissue, nor does it allow to distinguish the two main compartments of abdominal adipose tissue: visceral adipose tissue (VAT) and subcutaneous adipose tissue (SAT). Mounting evidence in recent years evaluated the automated abdominal adipose tissue segmentation on CT and MRI scans by means of machine learning and deep learning algorithms. On this respect, the Dice Score is a common metric to assess the spatial overlap between the predicted label maps and the ground truth. It provides both size and localization consensus for any type of method, not only AI.

Extending our previous theme Companies who had AI and Digital at their core have fared far bette..., in this post, we consider the approaches to AI from individual companies In a nutshell, what it says is .. companies in the AI space are choosing their favourite deep learning technique and focussing on it For example, the UK recently announced a UK National AI strategy which figures an emphasis on'Cyber physical strategy'. As the ancient proverb goes, if you are working with AI, we live interesting times, and that's both a blessing and a struggle. Like many in the industry, I love the complexity and the dynamism of AI!

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Let the OSS Enterprise newsletter guide your open source journey! In June, OpenAI teamed up with GitHub to launch Copilot, a service that provides suggestions for whole lines of code inside development environments like Microsoft Visual Studio. Powered by an AI model called Codex -- which OpenAI later exposed through an API -- Copilot can translate natural language into code across more than a dozen programming languages, interpreting commands in plain English and executing them. Now, a community effort is underway to create an open source, freely available alternative to Copilot and OpenAI's Codex model.

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Why would you need a robot with a ten-inch screen, camera, sensors, and a bunch of other gadgets to go around your home and make Wall-E noises? Because Amazon thinks it might be useful in the future. Astro, Amazon's latest innovation, looks a lot like an Echo Show on wheels. It packs a lot of interesting technology and shows just how far deep learning, sensor technology, and mobile robots have come.

The AI and Machine Learning industry is booming like never before. As of 2021, the increase in AI usage across businesses will create $2.9 trillion of business value. AI has automated many industries across the globe and changed the way they operate. Most large companies incorporate AI to maximize productivity in their workflow, and industries like marketing and healthcare have undergone a paradigm shift due to the consolidation of AI. Due to this, there has been an increasing demand in the past few years for AI professionals. There has almost been a 100% increase in AI and machine learning-related job postings from 2015 to 2018. This number has grown since and is projected to rise in 2021. If you are looking to break into the machine learning industry, the good news is that there is no shortage of jobs available. Companies need a talented workforce that is capable of pioneering the shift to machine learning.

Accurate weather prediction is essential for many aspects of life, notably the early warning of extreme weather events such as rainstorms. Short-term predictions of these events rely on forecasts from numerical weather models, in which, despite much improvement in the past decades, outstanding issues remain concerning model uncertainties, and increasing demands for computation and storage resources. In recent years, the advance of deep learning offers a viable alternative approach. Here, we show that a 3D convolutional neural network using a single frame of meteorology fields as input is capable of predicting the precipitation spatial distribution. The network is developed based on 39-years (1980-2018) data of meteorology and daily precipitation over the contiguous United States. The results bring fundamental advancements in weather prediction. First, the trained network alone outperforms the state-of-the-art weather models in predicting daily total precipitation, and the superiority of the network extends to forecast leads up to 5 days. Second, combining the network predictions with the weather-model forecasts significantly improves the accuracy of model forecasts, especially for heavy-precipitation events. Third, the millisecond-scale inference time of the network facilitates large ensemble predictions for further accuracy improvement. These findings strongly support the use of deep-learning in short-term weather predictions.

In spite of the enormous success of neural networks, adversarial examples remain a relatively weakly understood feature of deep learning systems. There is a considerable effort in both building more powerful adversarial attacks and designing methods to counter the effects of adversarial examples. We propose a method to transform the adversarial input data through a mixture of generators in order to recover the correct class obfuscated by the adversarial attack. A canonical set of images is used to generate adversarial examples through potentially multiple attacks. Such transformed images are processed by a set of generators, which are trained adversarially as a whole to compete in inverting the initial transformations. To our knowledge, this is the first use of a mixture-based adversarially trained system as a defense mechanism. We show that it is possible to train such a system without supervision, simultaneously on multiple adversarial attacks. Our system is able to recover class information for previously-unseen examples with neither attack nor data labels on the MNIST dataset. The results demonstrate that this multi-attack approach is competitive with adversarial defenses tested in single-attack settings.