Like the development of a car model (say, electrical cars), developing AI models is a costly and time-consuming process. The lifecycle of an AI model can be divided into two phases: training and deployment. The training phase includes data collection and pre-processing, model selection (e.g., architecture search and design), hyperparameter tuning, model parameter optimization, and validation. AI model training can be quite expensive, especially when it comes to the training of foundation models4 that require pre-training on large-scale datasets with neural networks consisting of a gigantic size of trainable parameters. Take the Generative Pre-trained Transformer 3 (GPT-3)5 as an example, which is one of the largest languages models ever trained to date.

Simply put, a language model is a statistical model that learns the distribution or probabilities of words in a sequence. It turns out that if we can achieve such a model with high fidelity, we can solve a few interesting tasks. For example, if we know that a word is likely to occur given some sequence of words, we can implement some useful functionality like email autocomplete (e.g., given the sequence "Have a great " .. we can predict that the next likely word is "day"). When these statistical models are derived using large neural networks with billions of parameters (hence the term large language models or LLMs), the results and application areas are even more impressive. Results from transformer-based model architectures like BERT, GPT etc., show that these models excel at several complex tasks e.g., they can mimic creative writing, predict sentiment, identify topics within sentences with few examples, meaningfully summarize lengthy documents, translate languages etc.

We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena. VALSE offers a suite of six tests covering various linguistic constructs. Solving these requires models to ground linguistic phenomena in the visual modality, allowing more fine-grained evaluations than hitherto possible. We build VALSE using methods that support the construction of valid foils, and report results from evaluating five widely-used V&L models. Our experiments suggest that current models have considerable difficulty addressing most phenomena. Hence, we expect VALSE to serve as an important benchmark to measure future progress of pretrained V&L models from a linguistic perspective, complementing the canonical task-centred V&L evaluations.

Want your bedroom to look like a "sunset in Paris"? A new AI project will help you find the right color scheme from a simple text description. Danny Richman's Free GPT-3 Color Palette Generator uses machine learning to pick out colors associated with text phrases. You might, for example, enter "baby nursery" or "New England" or "Martian sunset" and the AI will find a palette of four colors associated with that term. Richman's generator, which is built in Google Sheets and is free for anyone to use, also returns the HEX values, making it easy for web designers to apply those colors on their websites, for example.

In this new effort, the team at DeepMind set out to develop a tool to help in such endeavors.

Did you miss a session at the Data Summit? This week in AI, DARPA, the emerging technologies R&D agency of the U.S. Defense Department, launched a new program that aims to "align" AI systems with human decision-makers in domains where there isn't an agreed-upon right answer. Elsewhere, two prominent cofounders from LinkedIn and DeepMind, Reid Hoffman and Mustafa Suleyman, announced a new AI startup called Inflection AI that seeks to develop software that allows humans to talk to computers using everyday language. In a press release describing the new three-and-a-half-year program, DARPA says that the goal is to "evaluate and build trusted algorithmic decision-makers for mission-critical Department of Defense operations." Dubbed "In the Moment," or ITM, it focuses on the process of alignment -- building AI systems that accomplish what they're expected to accomplish.

One doesn't have to dig too deep into legal organizations to find AI skeptics. AI is getting tremendous attention and significant venture capital, but AI tools frequently underwhelm in the trenches. Here are a few reasons why that is and why I believe GPT-3, a beta version of which was recently released by the OpenAI Foundation, might be a game-changer in legal and other knowledge-focused organizations. GPT-3 is getting a lot of oxygen lately because of its size, scope, and capabilities. However, it should be recognized that a significant amount of that attention is due to its association with Elon Musk.

Overcoming the unidirectional limit while maintaining an independent masking algorithm based on permutation, XLNet improves upon the state-of-the-art autoregressive model that is TransformerXL. Using a bidirectional context while keeping its autoregressive approach, this model outperforms BERT on 20 tasks while keeping an impressive generative coherence.

Billionaire Reid Hoffman, DeepMind cofounder Mustafa Suleyman and former DeepMind researcher Karen Simonyan have announced the launch of a new company called Inflection AI. Suleyman will be appointed as CEO of the company. Inflection aims to develop a new suite of AI technologies that will ease communication between machines and humans. The company hasn't yet revealed the products and services that it will sell. In a statement released to the public, Suleyman noted, "Recent advances in artificial intelligence promise to fundamentally redefine human-machine interaction. We will soon have the ability to relay our thoughts and ideas to computers using the same natural, conversational language we use to communicate with people. Over time these new language capabilities will revolutionise what it means to have a digital experience."

DeepMind co-founder Mustafa Suleyman has joined two other high-profile industry figures in launching a new venture called Inflection AI. LinkedIn co-founder Reid Hoffman is joining Suleyman on the venture. "Reid and I are excited to announce that we are co-founding a new company, Inflection AI," wrote Suleyman in a statement. "Inflection will be an AI-first consumer products company, incubated at Greylock, with all the advantages and expertise that come from being part of one of the most storied venture capital firms in the world." Dr Karén Simonyan, another former DeepMind AI expert, will serve as Inflection AI's chief scientist and its third co-founder.