One of the biggest highlights of Build, Microsoft's annual software development conference, was the presentation of a tool that uses deep learning to generate source code for office applications. The tool uses GPT-3, a massive language model developed by OpenAI last year and made available to select developers, researchers, and startups in a paid application programming interface. Many have touted GPT-3 as the next-generation artificial intelligence technology that will usher in a new breed of applications and startups. Since GPT-3's release, many developers have found interesting and innovative uses for the language model. And several startups have declared that they will be using GPT-3 to build new or augment existing products. But creating a profitable and sustainable business around GPT-3 remains a challenge.

In the race to build the underlying technologies that can power the next wave of AI revolution, a Chinese lab just toppled OpenAI, the venerated US-based research lab, in terms of who can train a gigantic deep learning model with the most training parameters--as for whether or not there is a race, at least ranking members of the lab believe so. The Beijing Academy of Artificial Intelligence, styled as BAAI and known in Chinese as 北京智源人工智能研究院, launched the latest version of Wudao 悟道, a pre-trained deep learning model that the lab dubbed as "China's first," and "the world's largest ever," with a whopping 1.75 trillion parameters. Unlike conventional deep learning models that are usually task-specific, Wudao is a multi-modal model trained to tackle both text and image, two dramatically different sets of problems. At BAAI's annual academic conference on Tuesday, the institution demonstrated Wudao performing tasks such as natural language processing, text generation, image recognition, image generation, etc. The model is capable of writing poems and couplets in the traditional Chinese styles, answer questions, write essays, generate alt text for images, and generate corresponding images from natural language description with a decent level of photorealism. It is even able to power "virtual idols", with the help of XiaoIce, a Chinese company spun off of Microsoft--so there can be voice support too, in addition to text and image.

Naver, the Seongnam, South Korean-based company that operates the eponymous search engine Naver, this week announced that it trained one of the largest AI language models of its kind, called HyperCLOVA. Naver claims that the system learned 6,500 times more Korean data than OpenAI's GPT-3 and contains 204 billion parameters, the parts of the machine learning model learned from historical training data. For the better part of a year, OpenAI's GPT-3 has remained among the largest AI language models ever created. Via an API, people have used it to automatically write emails and articles, summarize text, compose poetry and recipes, create website layouts, and generate code for deep learning in Python. But GPT-3 has key limitations, chief among them that it's only available in English.

In May 2020 OpenAI presented GPT-3 in a paper titled Language Models are Few Shot Learners. GPT-3, the largest neural network ever created, revolutionized the AI world. OpenAI released a beta API for people to play with the system and soon the hype started building up. People were finding crazy results. GPT-3 could transform a description of a web page into the corresponding code.

Deep generative models are becoming widely used across science and industry for a variety of purposes. A common challenge is achieving a precise implicit or explicit representation of the data probability density. Recent proposals have suggested using classifier weights to refine the learned density of deep generative models. We extend this idea to all types of generative models and show how latent space refinement via iterated generative modeling can circumvent topological obstructions and improve precision. This methodology also applies to cases were the target model is non-differentiable and has many internal latent dimensions which must be marginalized over before refinement. We demonstrate our Latent Space Refinement (LaSeR) protocol on a variety of examples, focusing on the combinations of Normalizing Flows and Generative Adversarial Networks. We make all codes publicly available.

Exactly a year ago, OpenAI unveiled the GPT-3 with a whopping 175 billion parameters, which was made available to developers through an API in private beta. Since then, developers across the globe have been using GPT-3 to create realistic dialogues, summarise complex documents, customer service questions, and make search better than ever before. The company's decision to not open-source GPT-3 gave it more control over the use cases. However, in the recent past, there have been instances of GPT-3 going rogue, like in the case of GPT-3 Dungeon. Microsoft acquired an exclusive license to GPT-3 last year, in the wake of its $1 billion investment in OpenAI.

As AI has grown from a menagerie of research projects to include a handful of titanic, industry-powering models like GPT-3, there is a need for the sector to evolve -- or so thinks Dario Amodei, former VP of research at OpenAI, who struck out on his own to create a new company a few months ago. Anthropic, as it's called, was founded with his sister Daniela and its goal is to create "large-scale AI systems that are steerable, interpretable, and robust." The challenge the siblings Amodei are tackling is simply that these AI models, while incredibly powerful, are not well understood. GPT-3, which they worked on, is an astonishingly versatile language system that can produce extremely convincing text in practically any style, and on any topic. But say you had it generate rhyming couplets with Shakespeare and Pope as examples.

Microsoft will use OpenAI's GPT-3 language model and "other Microsoft AI technology" to generate Power Platform formulas, known as Power Fx, using natural language input from users. "Now you'll be able to simply tell Power Apps what you'd like to see--for example, 'show me customers from the US whose subscription expired'--and a set of formulas will be presented along with an explanation of how they work," explained Power Apps director of program management Ryan Cunningham. The preview for the new toolset, called Power Apps Ideas, is due in June and will be built into Power Apps Studio. Microsoft introduced Power Fx in March 2021 as a low-code programming language designed to eventually be used across all Power Platform tools. Microsoft invested $1 billion in an AI platform with OpenAI in 2019.

We train deep generative models on datasets of reflexive polytopes. This enables us to compare how well the models have picked up on various global properties of generated samples. Our datasets are complete in the sense that every single example, up to changes of coordinate, is included in the dataset. Using this property we also perform tests checking to what extent the models are merely memorizing the data. We also train models on the same dataset represented in two different ways, enabling us to measure which form is easiest to learn from. We use these experiments to show that deep generative models can learn to generate geometric objects with non-trivial global properties, and that the models learn some underlying properties of the objects rather than simply memorizing the data.

The OpenAI Startup Fund is investing $100 million to help AI companies have a profound, positive impact on the world. We're looking to partner with a small number of early-stage startups in fields where artificial intelligence can have a transformative effect--like health care, climate change, and education--and where AI tools can empower people by helping them be more productive. The fund is managed by OpenAI, with investment from Microsoft and other OpenAI partners. In addition to capital, companies in the OpenAI Startup Fund will get early access to future OpenAI systems, support from our team, and credits on Azure. If your startup plans to push the boundaries of today's artificial intelligence by building with our API, we want to hear from you.