This article will take you through what digital transformations are, what drives it, how to aid successful digital transformations, how AI and deep learning can help, the challenges you might face in implementation and how to work around them. We will also talk about what the current pace of technological growth means for the future of work and what we can do about the paranoia that goes along with increasing automation. While talking about singularity or Skynet taking over is not the point of this blog, it would be a little apathetic to not acknowledge the risks that come with acceleration in technological advancement. Have a data extraction problem in mind? Head over to Nanonets and start building models for free!

Transformer models have become the defacto standard for NLP tasks. As an example, I'm sure you've already seen the awesome GPT3 Transformer demos and articles detailing how much time and money it took to train. But even outside of NLP, you can also find transformers in the fields of computer vision and music generation. That said, for such a useful model, transformers are still very difficult to understand. It took me multiple readings of the Google research paper first introducing transformers, and a host of blog posts to really understand how transformers work. I'll try to keep the jargon and the technicality to a minimum, but do keep in mind that this topic is complicated. I'll also include some basic math and try to keep things light to ensure the long journey is fun. Q: Why should I understand Transformers? In the past, the state of the art approach to language modeling problems (put simply, predicting the next word) and translations systems was the LSTM and GRU architecture (explained here) along with the attention mechanism.

"Double descent does not happen through depth." The double descent curve, tells that increasing model capacity past the interpolation threshold can lead to a decrease in test error. Increasing neural network capacity through width leads to double descent. How does increase or reduction in-depth play out towards the end? A group of researchers from MIT have attempted to explore this question in their work titled, "Do Deeper Convolutional Networks Perform Better?".

Discussions: Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments) Translations: Chinese (Simplified), Japanese, Korean, Russian, Spanish Watch: MIT’s Deep Learning State of the Art lecture referencing this post In the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformers outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions. The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter. A High-Level Look Let’s begin by looking at the model as a single black box. In a machine translation application, it would take a sentence in one language, and output its translation in another.

Machine Learning is more than just neural networks. It involves various classification and regression algorithms like KNN, SVG, etc... It recently (since 10 years more or less) became more popular due to the success of neural networks and deep learning (deep neural networks) in a variety of tasks such as speech recognition, object detection (in images) etc... A neural network is just a function, or a mapping if you prefer, of some input to some output (not much different from a basic function y 2*x 5, just much more articulated). For example, Deep Neural Networks (Neural networks with many layers) can, given an image, tell you if a particular object you are interested in is present (in which case outputs 1) or not (0).

Google Colab is a project from Google Research, a free, Jupyter based environment that allows us to create Jupyter [programming] notebooks to write and execute Python [1](and other Python-based third-party tools and machine learning frameworks such as Pandas, PyTorch, Tensorflow, Keras, Monk, OpenCV, and others) in a web browser. A programming notebook is a type of a shell or kernel in the form of a word processor, where we can write and execute code. The data required for processing in Google Colab can be mounted into Google Drive or imported from any source on the internet. Project Jupyter is an open-source software organization that develops and supports Jupyter notebooks for interactive computing [4]. Google Colab requires no configuration to get started and provides free access to GPUs.

GPT-3 has been used for many different things and it is doing these things with unimaginable accuracy. This language model is although not very foolproof as it can generate chunks of results which are not comprehensible or understandable by humans. Now I am flabbergasted by the fact that GPT-3 can write movie scripts and has humour and twist incorporated in it. GPT-3 has written a script for the movie named Solicitors. The film was directed and shot by Jacob Vaus, produced by Eli Weiss.

The hype around AI is growing rapidly, as most research companies predict AI will take on an increasingly important role in the future. While business leaders are very interested in leveraging machine learning technology, there's a talent shortage standing in the way. It turns out that there are very few developers that have the skills needed to spearhead serious new AI projects. This means that developers who can acquire these skills will be highly in demand. With all this in mind, let's take a look at several facts about AI every developer should know before changing their focus to machine learning, artificial intelligence, and--while we're at it--deep learning and neural networks.

Production of news content is growing at an astonishing rate. To help manage and monitor the sheer amount of text, there is an increasing need to develop efficient methods that can provide insights into emerging content areas, and stratify unstructured corpora of text into `topics' that stem intrinsically from content similarity. Here we present an unsupervised framework that brings together powerful vector embeddings from natural language processing with tools from multiscale graph partitioning that can reveal natural partitions at different resolutions without making a priori assumptions about the number of clusters in the corpus. We show the advantages of graph-based clustering through end-to-end comparisons with other popular clustering and topic modelling methods, and also evaluate different text vector embeddings, from classic Bag-of-Words to Doc2Vec to the recent transformers based model Bert. This comparative work is showcased through an analysis of a corpus of US news coverage during the presidential election year of 2016.

Robustness to adversarial perturbations and accurate uncertainty estimation are crucial for reliable application of deep learning in real world settings. Dirichlet-based uncertainty (DBU) models are a family of models that predict the parameters of a Dirichlet distribution (instead of a categorical one) and promise to signal when not to trust their predictions. Untrustworthy predictions are obtained on unknown or ambiguous samples and marked with a high uncertainty by the models. In this work, we show that DBU models with standard training are not robust w.r.t. three important tasks in the field of uncertainty estimation. In particular, we evaluate how useful the uncertainty estimates are to (1) indicate correctly classified samples, and (2) to detect adversarial examples that try to fool classification. We further evaluate the reliability of DBU models on the task of (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. To this end, we present the first study of certifiable robustness for DBU models. Furthermore, we propose novel uncertainty attacks that fool models into assigning high confidence to OOD data and low confidence to ID data, respectively. Based on our results, we explore the first approaches to make DBU models more robust. We use adversarial training procedures based on label attacks, uncertainty attacks, or random noise and demonstrate how they affect robustness of DBU models on ID data and OOD data.