Naver, the Seongnam, South Korean-based company that operates the eponymous search engine Naver, this week announced that it trained one of the largest AI language models of its kind, called HyperCLOVA. Naver claims that the system learned 6,500 times more Korean data than OpenAI's GPT-3 and contains 204 billion parameters, the parts of the machine learning model learned from historical training data. For the better part of a year, OpenAI's GPT-3 has remained among the largest AI language models ever created. Via an API, people have used it to automatically write emails and articles, summarize text, compose poetry and recipes, create website layouts, and generate code for deep learning in Python. But GPT-3 has key limitations, chief among them that it's only available in English.

In May 2020 OpenAI presented GPT-3 in a paper titled Language Models are Few Shot Learners. GPT-3, the largest neural network ever created, revolutionized the AI world. OpenAI released a beta API for people to play with the system and soon the hype started building up. People were finding crazy results. GPT-3 could transform a description of a web page into the corresponding code.

Deep generative models are becoming widely used across science and industry for a variety of purposes. A common challenge is achieving a precise implicit or explicit representation of the data probability density. Recent proposals have suggested using classifier weights to refine the learned density of deep generative models. We extend this idea to all types of generative models and show how latent space refinement via iterated generative modeling can circumvent topological obstructions and improve precision. This methodology also applies to cases were the target model is non-differentiable and has many internal latent dimensions which must be marginalized over before refinement. We demonstrate our Latent Space Refinement (LaSeR) protocol on a variety of examples, focusing on the combinations of Normalizing Flows and Generative Adversarial Networks. We make all codes publicly available.

Exactly a year ago, OpenAI unveiled the GPT-3 with a whopping 175 billion parameters, which was made available to developers through an API in private beta. Since then, developers across the globe have been using GPT-3 to create realistic dialogues, summarise complex documents, customer service questions, and make search better than ever before. The company's decision to not open-source GPT-3 gave it more control over the use cases. However, in the recent past, there have been instances of GPT-3 going rogue, like in the case of GPT-3 Dungeon. Microsoft acquired an exclusive license to GPT-3 last year, in the wake of its $1 billion investment in OpenAI.

As AI has grown from a menagerie of research projects to include a handful of titanic, industry-powering models like GPT-3, there is a need for the sector to evolve -- or so thinks Dario Amodei, former VP of research at OpenAI, who struck out on his own to create a new company a few months ago. Anthropic, as it's called, was founded with his sister Daniela and its goal is to create "large-scale AI systems that are steerable, interpretable, and robust." The challenge the siblings Amodei are tackling is simply that these AI models, while incredibly powerful, are not well understood. GPT-3, which they worked on, is an astonishingly versatile language system that can produce extremely convincing text in practically any style, and on any topic. But say you had it generate rhyming couplets with Shakespeare and Pope as examples.

Microsoft will use OpenAI's GPT-3 language model and "other Microsoft AI technology" to generate Power Platform formulas, known as Power Fx, using natural language input from users. "Now you'll be able to simply tell Power Apps what you'd like to see--for example, 'show me customers from the US whose subscription expired'--and a set of formulas will be presented along with an explanation of how they work," explained Power Apps director of program management Ryan Cunningham. The preview for the new toolset, called Power Apps Ideas, is due in June and will be built into Power Apps Studio. Microsoft introduced Power Fx in March 2021 as a low-code programming language designed to eventually be used across all Power Platform tools. Microsoft invested $1 billion in an AI platform with OpenAI in 2019.

We train deep generative models on datasets of reflexive polytopes. This enables us to compare how well the models have picked up on various global properties of generated samples. Our datasets are complete in the sense that every single example, up to changes of coordinate, is included in the dataset. Using this property we also perform tests checking to what extent the models are merely memorizing the data. We also train models on the same dataset represented in two different ways, enabling us to measure which form is easiest to learn from. We use these experiments to show that deep generative models can learn to generate geometric objects with non-trivial global properties, and that the models learn some underlying properties of the objects rather than simply memorizing the data.

The OpenAI Startup Fund is investing $100 million to help AI companies have a profound, positive impact on the world. We're looking to partner with a small number of early-stage startups in fields where artificial intelligence can have a transformative effect--like health care, climate change, and education--and where AI tools can empower people by helping them be more productive. The fund is managed by OpenAI, with investment from Microsoft and other OpenAI partners. In addition to capital, companies in the OpenAI Startup Fund will get early access to future OpenAI systems, support from our team, and credits on Azure. If your startup plans to push the boundaries of today's artificial intelligence by building with our API, we want to hear from you.

In recent years, researchers have used artificial intelligence to improve translation between programming languages or automatically fix problems. The AI system DrRepair, for example, has been shown to solve most issues that spawn error messages. But some researchers dream of the day when AI can write programs based on simple descriptions from non-experts. On Tuesday, Microsoft and OpenAI shared plans to bring GPT-3, one of the world's most advanced models for generating text, to programming based on natural language descriptions. This is the first commercial application of GPT-3 undertaken since Microsoft invested $1 billion in OpenAI last year and gained exclusive licensing rights to GPT-3.

OpenAI is launching a $100 million startup fund, which it calls the OpenAI Startup Fund, though which it and its partners will invest in early-stage AI companies tackling major problems (and productivity). Among those partners and investors in the fund is Microsoft, at whose Build conference OpenAI founder Sam Altman announced the news. In a prerecorded video, Altman explained that "this is not a typical corporate venture fund. We plan to make big early bets on a relatively small number of companies, probably not more than 10." It's not clear exactly how the $100M will be divided or disbursed, or on what timeline, or whether this is part of a longer program. But it seems to be a limited fund, not just the 2021 round. Altman did say that they will be looking for companies that are taking on serious issues, like healthcare, climate change, and education, where AI-powered applications or approaches could "benefit all of humanity," in keeping with OpenAI's mission statement.