Collaborating Authors

Accelerate Your First AI Deployment with Pre-Trained Models


Organizations are constantly looking to incorporate artificial intelligence (AI) in their daily operations. However, with the large amounts of time and money required for extensive AI integration, organizations must find smarter ways to implement AI, such as using pre-trained AI models. As you probably know, transforming an organization with AI and machine learning can be time-consuming. The efforts and finances required to complete the process depend on the level of automation and digitization being introduced in the various departments of the organization. Deep learning and other components of AI need thousands upon thousands of datasets to improve the competency of automated operations over a given period.

LogME: Practical Assessment of Pre-trained Models for Transfer Learning Artificial Intelligence

This paper studies task adaptive pre-trained model selection, an \emph{underexplored} problem of assessing pre-trained models so that models suitable for the task can be selected from the model zoo without fine-tuning. A pilot work~\cite{nguyen_leep:_2020} addressed the problem in transferring supervised pre-trained models to classification tasks, but it cannot handle emerging unsupervised pre-trained models or regression tasks. In pursuit of a practical assessment method, we propose to estimate the maximum evidence (marginalized likelihood) of labels given features extracted by pre-trained models. The maximum evidence is \emph{less prone to over-fitting} than the likelihood, and its \emph{expensive computation can be dramatically reduced} by our carefully designed algorithm. The Logarithm of Maximum Evidence (LogME) can be used to assess pre-trained models for transfer learning: a pre-trained model with high LogME is likely to have good transfer performance. LogME is fast, accurate, and general, characterizing it as \emph{the first practical assessment method for transfer learning}. Compared to brute-force fine-tuning, LogME brings over $3000\times$ speedup in wall-clock time. It outperforms prior methods by a large margin in their setting and is applicable to new settings that prior methods cannot deal with. It is general enough to diverse pre-trained models (supervised pre-trained and unsupervised pre-trained), downstream tasks (classification and regression), and modalities (vision and language). Code is at \url{}.

3 Pre-Trained Model Series to Use for NLP with Transfer Learning


Before we start, if you are reading this article, I am sure that we share similar interests and are/will be in similar industries. So let's connect via Linkedin! Please do not hesitate to send a contact request! If you have been trying to build machine learning models with high accuracy; but never tried Transfer Learning, this article will change your life. At least, it did mine!

FROM Pre-trained Word Embeddings TO Pre-trained Language Models -- Focus on BERT


Language modeling is the task of assigning a probability distribution over sequences of words that matches the distribution of a language. Although it sounds formidable, language modeling (i.e. ELMo, BERT, GPT) is essentially just predicting words in a blank. More formally, given a context, a language model predicts the probability of a word occurring in that context. Why is this method effective?

AI in 2019 will be all about bots and pre-trained models


This past year was big for voice assistants. With significant product releases from Amazon, Apple, Microsoft, Google, Samsung, Baidu and others, vendors are flooding the market with these products so that people can become more comfortable with conversational modes of interaction. While mostly focused on the consumer audience, it's clear that vendors are shifting their attention to the enterprise. Just as 2018 was the year of consumer voice assistant overload, 2019 will be the year that enterprises will see widespread adoption and implementation of voice assistants. Already, companies are realizing the benefits of AI-based conversational technologies as extensions of their business to support a wide range of tasks.