The most impressive thing about OpenAI's natural language processing (NLP) model, GPT-3, is its sheer size. With more than 175 billion weighted connections between words known as parameters, the transformer encoder-decoder model blows its 1.5 billion parameter predecessor, GPT-2, out of the water. This has allowed the model to generate text that is surprisingly human-like after only being fed a few examples of the task you want it to do. Its release in 2020 dominated headlines, and people were scrambling to get on the waitlist to access its API hosted on OpenAI's cloud service. Now, months later, as more users have gained access to the API (myself included), interesting applications and use cases have been popping up every day.

The machine learning company OpenAI is developing models that improve computer vision and can produce original images from a text prompt. Why it matters: The new models are the latest steps in ongoing efforts to create machine learning systems that exhibit elements of general intelligence, while performing tasks that are actually useful in the real world -- without breaking the bank on computing power. What's happening: OpenAI today is announcing two new systems that attempt to do for images what its landmark GPT-3 model did last year for text generation. What they're saying: "Last year, we were able to make substantial progress on text with GPT-3, but the thing is that the world isn't just built on text," says Sutskever. "This is a step towards the grander goal of building a neural network that can work in both images and text."

DALL-E from Open AI can generate images from text captions. It is very similar to GPT-3 and Image GPT and produces amazing results.

As you can see, GPT-3 is capable of producing complex sentences that read as though they could have been produced by a human. The example sentences include cultural references and a believable account of how the scientists would react. Machines that can use language in this way are important for several reasons. Language is crucial to making sense of the everyday world: humans use it to communicate, to share ideas and describe concepts. An AI that mastered language would acquire a better understanding of the world in the process.

Last July, GPT-3 took the internet by storm. The massive 175 billion-parameter autoregressive language model, developed by OpenAI, showed a startling ability to translate languages, answer questions, and – perhaps most eerily – generate its own coherent passages, poems, and songs when given examples to process. As it turns out, experts were captivated by these abilities, too: captivated enough, in fact, that researchers from OpenAI and a number of universities met several months ago to discuss the technical and sociopolitical implications of the platform. The summit, helmed by OpenAI in partnership with Stanford's Institute for Human-Centered Artificial Intelligence, convened in October. Apart from those two institutions, the remainder of the participants are currently unknown by the public, as the meeting was held under the Chatham House Rule, whereby a meeting's information is public but its participants are secret.

GPT-3 is the world's most sophisticated natural language technology. Discover how companies are implementing the OpenAI GPT-3 API to power new use cases.

We consider image completion from the perspective of amortized inference in an image generative model. We leverage recent state of the art variational auto-encoder architectures that have been shown to produce photo-realistic natural images at non-trivial resolutions. Through amortized inference in such a model we can train neural artifacts that produce diverse, realistic image completions even when the vast majority of an image is missing. We demonstrate superior sample quality and diversity compared to prior art on the CIFAR-10 and FFHQ-256 datasets. We conclude by describing and demonstrating an application that requires an in-painting model with the capabilities ours exhibits: the use of Bayesian optimal experimental design to select the most informative sequence of small field of view x-rays for chest pathology detection.

Reinforcement learning (RL) is one of the most active fields of AI research. Despite the interest demonstrated by the research community in reinforcement learning, the development methodology still lags behind, with a severe lack of standard APIs to foster the development of RL applications. OpenAI Gym is probably the most used environment to develop RL applications and simulations, but most of the abstractions proposed in such a framework are still assuming a semi-structured methodology. This is particularly relevant for agent-based models whose purpose is to analyse adaptive behaviour displayed by self-learning agents in the simulation. In order to bridge this gap, we present a workflow and tools for the decoupled development and maintenance of multi-purpose agent-based models and derived single-purpose reinforcement learning environments, enabling the researcher to swap out environments with ones representing different perspectives or different reward models, all while keeping the underlying domain model intact and separate. The Sim-Env Python library generates OpenAI-Gym-compatible reinforcement learning environments that use existing or purposely created domain models as their simulation back-ends. Its design emphasizes ease-of-use, modularity and code separation.

Estimation of spatially-varying parameters for computationally expensive forward models governed by partial differential equations is addressed. A novel multiscale Bayesian inference approach is introduced based on deep probabilistic generative models. Such generative models provide a flexible representation by inferring on each scale a low-dimensional latent encoding while allowing hierarchical parameter generation from coarse- to fine-scales. Combining the multiscale generative model with Markov Chain Monte Carlo (MCMC), inference across scales is achieved enabling us to efficiently obtain posterior parameter samples at various scales. The estimation of coarse-scale parameters using a low-dimensional latent embedding captures global and notable parameter features using an inexpensive but inaccurate solver. MCMC sampling of the fine-scale parameters is enabled by utilizing the posterior information in the immediate coarser-scale. In this way, the global features are identified in the coarse-scale with inference of low-dimensional variables and inexpensive forward computation, and the local features are refined and corrected in the fine-scale. The developed method is demonstrated with two types of permeability estimation for flow in heterogeneous media. One is a Gaussian random field (GRF) with uncertain length scales, and the other is channelized permeability with the two regions defined by different GRFs. The obtained results indicate that the method allows high-dimensional parameter estimation while exhibiting stability, efficiency and accuracy.

DALL·E[1] is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs. We've found that it has a diverse set of capabilities, including creating anthropomorphized versions of animals and objects, combining unrelated concepts in plausible ways, rendering text, and applying transformations to existing images. GPT-3 showed that language can be used to instruct a large neural network to perform a variety of text generation tasks. Image GPT showed that the same type of neural network can also be used to generate images with high fidelity. We extend these findings to show that manipulating visual concepts through language is now within reach.