Microsoft on Tuesday announced an artificial intelligence system that can recommend code for software developers to use as they write code. Microsoft is looking to simplify the process of programming, the area where the company got its start in 1975. That could keep programmers who already use the company's tools satisfied and also attract new ones. The system, called GitHub Copilot, draws on source code uploaded to code-sharing service GitHub, which Microsoft acquired in 2018, as well as other websites. Microsoft and GitHub developed it with help from OpenAI, an AI research start-up that Microsoft backed in 2019.

In this special guest feature, Sahar Mor, founder of AirPaper, discusses DALL-E – a new powerful API from OpenAI that creates images from text captions. With this, Sahar is planning to build a few products such as a chart generator based on text and a text-based tool to generate illustrations for landing pages. Sahar has 12 years of Engineering Product Management experience, both focused on products with AI in their core. Previously he worked as an Engineering Manager in early-stage startups and at the elite Israeli intelligence unit – 8200. Several months ago OpenAI published their latest research model DALL-E – an advanced neural network that generates images from text prompts and a natural progression of its powerful language model GPT-3.

The project was born in July 2020 as a quest to replicate OpenAI GPT-family models. A group of researchers and engineers decided to give OpenAI a "run for their money" and so the project began. Their ultimate goal is to replicate GPT-3-175B to "break OpenAI-Microsoft monopoly" on transformer-based language models. Since the transformer was invented in 2017, we've seen increased effort in creating powerful language models. GPT-3 is the one that became a superstar, but all over the world companies and institutions are competing to find an edge that allows them to take a breath at a hegemonic position.

Generative Pre-trained Transformer 3 (GPT-3) embraces and augments the GPT-2 model architecture, including pre-normalization, modified initialization, and reversible tokenization. It exhibits strong performance on many Natural Language Processing (NLP) tasks. GPT-3 is an auto-regressive artificial intelligence algorithm developed by OpenAI, an AI-powered research laboratory located in San Francisco, California. It is a massive artificial neural network that takes help from deep learning to generate human-like text and is trained on huge text datasets with thousands of billions of words. It is the third-generation AI language prediction model in the GPT-n series and the successor to GPT-2. In simple words, OpenAI GPT-3 was fed inputs the ways how billions of people write and also was taught how to pick up on writing patterns based on user entry.

In July 2020, OpenAI launched GPT-3, an artificial intelligence language model that quickly stoked excitement about computers writing poetry, news articles, and programming code. Just as quickly, it was shown to sometimes be foulmouthed and toxic. OpenAI said it was working on fixes, but the company recently discovered GPT-3 was being used to generate child porn. Now OpenAI researchers say they've found a way to curtail GPT-3's toxic text by feeding the program roughly 100 encyclopedia-like samples of writing by human professionals on topics like history and technology but also abuse, violence, and injustice. OpenAI's project shows how the tech industry is scrambling to constrain the dark side of a technology that's shown enormous potential but also can spread disinformation and perpetuate biases.

Much scientific enquiry across disciplines is founded upon a mechanistic treatment of dynamic systems that ties form to function. A highly visible instance of this is in molecular biology, where an important goal is to determine functionally-relevant forms/structures that a protein molecule employs to interact with molecular partners in the living cell. This goal is typically pursued under the umbrella of stochastic optimization with algorithms that optimize a scoring function. Research repeatedly shows that current scoring function, though steadily improving, correlate weakly with molecular activity. Inspired by recent momentum in generative deep learning, this paper proposes and evaluates an alternative approach to generating functionally-relevant three-dimensional structures of a protein. Though typically deep generative models struggle with highly-structured data, the work presented here circumvents this challenge via graph-generative models. A comprehensive evaluation of several deep architectures shows the promise of generative models in directly revealing the latent space for sampling novel tertiary structures, as well as in highlighting axes/factors that carry structural meaning and open the black box often associated with deep models. The work presented here is a first step towards interpretative, deep generative models becoming viable and informative complementary approaches to protein structure prediction.

The spectacular success of deep generative models calls for quantitative tools to measure their statistical performance. Divergence frontiers have recently been proposed as an evaluation framework for generative models, due to their ability to measure the quality-diversity trade-off inherent to deep generative modeling. However, the statistical behavior of divergence frontiers estimated from data remains unknown to this day. In this paper, we establish non-asymptotic bounds on the sample complexity of the plug-in estimator of divergence frontiers. Along the way, we introduce a novel integral summary of divergence frontiers. We derive the corresponding non-asymptotic bounds and discuss the choice of the quantization level by balancing the two types of approximation errors arisen from its computation. We also augment the divergence frontier framework by investigating the statistical performance of smoothed distribution estimators such as the Good-Turing estimator. We illustrate the theoretical results with numerical examples from natural language processing and computer vision.

In a study published today, OpenAI, the lab best known for its research on large language models, claims it's discovered a way to improve the "behavior" of language models with respect to ethical, moral, and societal values. The approach, OpenAI says, can give developers the tools to dictate the tone and personality of a model depending on the prompt that the model's given. Despite the potential of natural language models like GPT-3, many blockers exist. The models can't always answer math problems correctly or respond to questions without paraphrasing training data, and it's well-established that they amplify the biases in data on which they were trained. That's problematic in the language domain, because a portion of the data is often sourced from communities with pervasive gender, race, and religious prejudices.

To help you fast track your company's adoption of machine learning (ML), AWS offers educational solutions for developers to get hands-on experience. We like to think of these programs as a fun way for developers to build their skills using ML technologies in real world scenarios. In this post, we walk you through how to prepare for and run an AI music competition using AWS DeepComposer. Through AWS DeepComposer, you can experience Generative AI in action and learn how to harness the latest in ML and AI. We provide an end-to-end kit that contains tools, techniques, processes, and best practices to run the event. Designed specifically to educate developers on generative AI, AWS DeepComposer includes tutorials, sample code, and training data in an immersive platform that can be used to build ML models with music as the medium of instruction. Developers, regardless of their background in ML or music, can get started with applying AI techniques including Generative Adversarial Networks (GANs), Autoregressive Convolutional Neural Networks (AR-CNN) and Transformers to generate new musical notes and accompaniments.

It's the worry that creeps in whenever people write about GPT-3: could this be used for bad? We've covered the technological advances in AI text generation like those from OpenAI a lot. There are always the "oohs" and "aah" about what it can do (write a self-help blog, for instance) and then others pointing out what it can't. But in the background is the question of how the ability to instantaneously "write" large amounts of text based on certain prompts could change the internet in unstoppable ways. Can artificial intelligence like GPT-3 be used for something like misinformation?