Never play games with a bot – it will find a way to cheat if they can. A team from OpenAI, an artificial intelligence lab in San Francisco co-founded by Elon Musk, has developed artificially intelligent bots that taught themselves to cooperate by playing hide-and-seek. The bots also learned how to use basic tools and that defying the laws of physics can help you win. In April, a team of bots known as the OpenAI Five beat the human world champions at the team-based video game DOTA 2. The hide-and-seek bots use similar principles to learn but the simpler game allows for more inventive play. Bowen Baker at OpenAI and his colleagues wanted to see if the team-based dynamics of the OpenAI Five could be used to generate skills that could one day be useful to humans.

The more I learn, the less I realize I know. This blog is the Part-2 of the series on reinforcement learning. Feel free to read the part-1 here. In this article I will be implementing OpenAI Gym's Bipedal Walker environment using Deep Deterministic Policy Gradient (DDPG) algorithm. OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms.

Energy-Based Models(EBM) is one of the most promising areas of deep learning that hasn't seen a tremendous level of adoption yet. Conceptually, EBMs are a form of generative modeling that learns the key characteristics of a target dataset and tries to generate similar datasets. While EBMs results appealing because of its simplicity they have experienced many challenges when applied in real world applications. Recently, AI-powerhouse OpenAI published a new research paper that explores a new technique to create EBM model that can scale across complex deep learning topologies. EBMs are typically used in one of the most complex problems of real world deep learning solutions: generating quality training datasets.

In February 2019, OpenAI released a paper describing GPT-2, a AI-based text-generation model based on the Transformer architecture and trained on massive amounts of text all around the internet. From a text-generation perspective, the included demos were very impressive: the text is coherent over a long horizon, and grammatical syntax and punctuation are near-perfect. At the same time, the Python code which allowed anyone to download the model (albeit smaller versions out of concern the full model can be abused to mass-generate fake news) and the TensorFlow code to load the downloaded model and generate predictions was open-sourced on GitHub. Neil Shepperd created a fork of OpenAI's repo which contains additional code to allow finetuning the existing OpenAI model on custom datasets. A notebook was created soon after, which can be copied into Google Colaboratory and clones Shepperd's repo to finetune GPT-2 backed by a free GPU.

Cryo-electron microscopy (cryo-EM) is a powerful technique for determining the structure of proteins and other macromolecular complexes at near-atomic resolution. In single particle cryo-EM, the central problem is to reconstruct the three-dimensional structure of a macromolecule from $10^{4-7}$ noisy and randomly oriented two-dimensional projections. However, the imaged protein complexes may exhibit structural variability, which complicates reconstruction and is typically addressed using discrete clustering approaches that fail to capture the full range of protein dynamics. Here, we introduce a novel method for cryo-EM reconstruction that extends naturally to modeling continuous generative factors of structural heterogeneity. This method encodes structures in Fourier space using coordinate-based deep neural networks, and trains these networks from unlabeled 2D cryo-EM images by combining exact inference over image orientation with variational inference for structural heterogeneity. We demonstrate that the proposed method, termed cryoDRGN, can perform ab initio reconstruction of 3D protein complexes from simulated and real 2D cryo-EM image data. To our knowledge, cryoDRGN is the first neural network-based approach for cryo-EM reconstruction and the first end-to-end method for directly reconstructing continuous ensembles of protein structures from cryo-EM images.

OpenAI, the artificial intelligence firm that Elon Musk founded then later departed, just released a stronger version of its "conversational" text-writing AI system. When OpenAI first released the algorithm, dubbed GPT-2, back in February, the company declared that it was too dangerous to release to the public, instead opting to share a watered-down version. Now, OpenAI announced that it's sharing a new version that's six times as robust as the original -- while keeping an eye out to make sure people don't misuse it. In the past, OpenAI expressed concerns that its AI could be used to flood the internet with fake news and propaganda. The first model had notable flaws and telltale signs that its output was machine-written.

It is now table stakes for artificial intelligence algorithms to "learn" about the world around them. The next level: For AI bots to learn how to talk to each other -- and develop their own shared language. New research released last week by OpenAI, the artificial intelligence nonprofit lab founded by Elon Musk and Y Combinator president Sam Altman, details how they're training AI bots to create their own language, based on trial and error, as the bots move around a set environment. This is different from how artificial intelligence algorithms typically learn -- using large sets of data, like to recognize a dog by taking in thousands of pictures of dogs. The world the researchers created for the AI bots to learn in is a computer simulation of a simple, two-dimensional white square.

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. Last week, artificial intelligence research lab OpenAI decided to release a more expanded version of GPT-2, the controversial text-generating AI model it first introduced in February. At the time, the lab refrained from releasing the full AI model, fearing it would be used for malicious purposes. Instead, OpenAI opted for a staged release of the AI, starting with a limited model (124 million parameters), and gradually releasing more capable models. In May, the research lab released the 355-million-parameter version of GPT-2, and last week, it finally released the 774-million-model, at 50 percent capacity of the text generator.

Last Thursday two lowly masters grad students, Aaron Gokaslan and Vanya Cohen managed to replicate the secretive OpenAI model and cheekily named their version OpenGPT-2. The code can be downloaded from this Google Colab page and apparently no prior experience in language modeling is required to use it. More useful might be the skills required to persuade Google to part with $50,000 worth of free cloud compute time for the training! Research firm OpenAI released a new, ever more powerful, version of their GPT language model with 1.5 billion parameters, trained on a data-set of 8 million web pages and although it's most entertaining use is to produce gibberish, it will inevitably also be able to produce coherent text sometime very soon. For us mere mortals, there's a cut down version of the model hosted in the cloud and a webpage that we can visit, type in a short phrase to prompt the system, and print out a few paragraphs of fake news.

I expect to suffer some degree of writer's block pretty much every day for the rest of my life. I'm a journalist and a novelist; it comes with the territory. But I have a feeling I'm going to suffer less from now on, thanks to my new best friend, GPT-2. Let me back up a bit: Six months ago, the research lab OpenAI created an AI system that generates text -- from fake news to poetry -- that in some cases actually sounds like it's written by a human being. The OpenAI team has been rolling it out in stages, each time giving us a more powerful version of the language model they dubbed GPT-2, and carefully watching to see how we use it.