Generative AI allows computers to learn fundamental patterns relevant to input, which is then used to manufacture similar content. This is achieved through generative adversarial networks (GANs), variational autoencoders, and transformers. Autoencoders help people automatically encode data and are made up of two distinct components, an encoder, and a decoder. Autoencoders reside in unsupervised artificial neural networks that memorize and quickly encode data that can then be reconstructed at a later date. A general adversarial network (GAN) is a type of machine learning framework that places two neural networks in a contest.

We've seen AI generate images from other images using GANs. Then, there were models able to generate questionable images using text. In early 2021, DALL-E was published, beating all previous attempts to generate images from text input using CLIP, a model that links images with text as a guide. A very similar task called image captioning may sound really simple but is, in fact, just as complex. It is the ability of a machine to generate a natural description of an image.

In this work we propose a deep adaptive sampling (DAS) method for solving partial differential equations (PDEs), where deep neural networks are utilized to approximate the solutions of PDEs and deep generative models are employed to generate new collocation points that refine the training set. The overall procedure of DAS consists of two components: solving the PDEs by minimizing the residual loss on the collocation points in the training set and generating a new training set to further improve the accuracy of current approximate solution. In particular, we treat the residual as a probability density function and approximate it with a deep generative model, called KRnet. The new samples from KRnet are consistent with the distribution induced by the residual, i.e., more samples are located in the region of large residual and less samples are located in the region of small residual. Analogous to classical adaptive methods such as the adaptive finite element, KRnet acts as an error indicator that guides the refinement of the training set. Compared to the neural network approximation obtained with uniformly distributed collocation points, the developed algorithms can significantly improve the accuracy, especially for low regularity and high-dimensional problems. We present a theoretical analysis to show that the proposed DAS method can reduce the error bound and demonstrate its effectiveness with numerical experiments.

When Microsoft injected $1 billion in artificial intelligence project co-founded by Elon Musk called OpenAI, it changed everything for the project. It brought the AI moonshot into the limelight and made much of its research more concrete. It's hard to catch how quickly things moved, but in November, 2021 something incredible happend amid all the headlines at OpenAI. That's when Microsoft launched the Azure OpenAI Service (November, 2021) -- giving Azure customers the ability to utilize OpenAI's machine learning models. Unveiled at the company's recent Ignite event, the service is based on GPT-3, a language model developed by OpenAI, the company which Microsoft backed with $1bn in 2019.

Even OpenAI is feeling the holiday spirit: they open sourced their photorealistic GLIDE model several days ago. Abhishek maps boring model diagrams to code for building intuition! JellyFish is a library for approximate & phonetic matching of strings. A new summarization evaluation metric called the Shannon Score is proposed. It performs the Shannon Game with a language model.

Large language models capable of writing poems, summaries, and computer code are driving the demand for "natural language processing (NLP) as a service." As these models become more capable -- and accessible, relatively speaking -- appetite in the enterprise for them is growing. According to a 2021 survey from John Snow Labs and Gradient Flow, 60% of tech leaders indicated that their NLP budgets grew by at least 10% compared to 2020, while a third -- 33% -- said that their spending climbed by more than 30%. Well-resourced providers like OpenAI, Cohere, and AI21 Labs are reaping the benefits. As of March, OpenAI said that GPT-3 was being used in more than 300 different apps by "tens of thousands" of developers and producing 4.5 billion words per day.

OpenAI is committed to developing general-purpose artificial intelligence that benefits all humanity, and we believe that achieving our goal requires expertise in public policy as well as technology. Will served three terms in the U.S. House of Representatives, has been a leading voice on technology policy, and coauthored bipartisan legislation outlining a national strategy for artificial intelligence. "Will brings a rare combination of expertise--he deeply understands both artificial intelligence as well as public policy, both of which are critical to a successful future for AI," said Sam Altman, OpenAI's CEO. Greg Brockman, OpenAI's chairman and Chief Technology Officer, added, "'AI public policy expert' isn't exactly a common title, and Will is squarely one of the leading ones. "I've been blown away by the scientific advances made by the team at OpenAI, and I've been inspired by their commitment to developing AI responsibly," said Will Hurd. OpenAI is committed to developing general-purpose artificial intelligence that benefits all humanity, and we believe that achieving our goal requires expertise in public policy as well as technology. Will served three terms in the U.S. House of Representatives, has been a leading voice on technology policy, and coauthored bipartisan legislation outlining a national strategy for artificial intelligence. "Will brings a rare combination of expertise--he deeply understands both artificial intelligence as well as public policy, both of which are critical to a successful future for AI," said Sam Altman, OpenAI's CEO. Greg Brockman, OpenAI's chairman and Chief Technology Officer, added, "'AI public policy expert' isn't exactly a common title, and Will is squarely one of the leading ones.

When Microsoft injected $1 billion in artificial intelligence project co-founded by Elon Musk called OpenAI, it changed everything for the project. It brought the AI moonshot into the limelight and made much of its research more concrete. It's hard to catch how quickly things moved, but in November, 2021 something incredible happend amid all the headlines at OpenAI. That's when Microsoft launched the Azure OpenAI Service (November, 2021) -- giving Azure customers the ability to utilize OpenAI's machine learning models. Unveiled at the company's recent Ignite event, the service is based on GPT-3, a language model developed by OpenAI, the company which Microsoft backed with $1bn in 2019.

Text-to-image generation has been one of the most active and exciting AI fields of 2021. In January, OpenAI introduced DALL-E, a 12-billion parameter version of the company's GPT-3 transformer language model designed to generate photorealistic images using text captions as prompts. An instant hit in the AI community, DALL-E's stunning performance also attracted widespread mainstream media coverage. Last month, tech giant NVIDIA released the GAN-based GauGAN2 -- the name taking inspiration from French Post-Impressionist painter Paul Gauguin as DALL-E had from Surrealist artist Salvador Dali. Not to be outdone, OpenAI researchers this week presented GLIDE (Guided Language-to-Image Diffusion for Generation and Editing), a diffusion model that achieves performance competitive with DALL-E while using less than one-third of the parameters.

In May 2020, OpenAI released a report on algorithmic efficiency improvements in deep learning. Main headline: Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period). Our results suggest that for AI tasks with high levels of recent investment, algorithmic progress has yielded more gains than classical hardware efficiency. A lot people were surprised by this; there’s a common narrative in which AI progress has come mostly from throwing more and more compute at relatively-dumb algorithms. (This is a common interpretation of The Bitter Lesson, though I would argue it is largely a misinterpretation.) I’ve had various experiences over the years which made the result not-that-surprising. Algorithms beating compute is the sort of thing I expect by default, on a gut level. The point of this post is to tell a few of the stories which underlie that intuition, aimed especially toward people who don’t have much first-hand experience with software engineering, ML, or simulation. (There will still be some jargon, though.) Disclaimer: this does not mean that you should put tons of confidence on this view. The goal is just to provide a possible lens through which “algorithmic progress has yielded more gains than classical hardware efficiency” makes sense; I want to raise that hypothesis from entropy. I’m not going to provide the sort of evidence which would justify very high confidence, I’m just going to point it out as a hypothesis to keep in the back of your mind, and update on when results like OpenAI’s come along. REWRITE IN C Back in college, I spent a summer simulating an unusual type of biochemical oscillator, officially under the aegis of the Minnesota Supercomputing Institute. The algorithm was conceptually simple: every time a reaction occurs between two molecules, update the counts of each molecule, then randomly sample to figure out when th