Last Thursday two lowly masters grad students, Aaron Gokaslan and Vanya Cohen managed to replicate the secretive OpenAI model and cheekily named their version OpenGPT-2. The code can be downloaded from this Google Colab page and apparently no prior experience in language modeling is required to use it. More useful might be the skills required to persuade Google to part with $50,000 worth of free cloud compute time for the training! Research firm OpenAI released a new, ever more powerful, version of their GPT language model with 1.5 billion parameters, trained on a data-set of 8 million web pages and although it's most entertaining use is to produce gibberish, it will inevitably also be able to produce coherent text sometime very soon. For us mere mortals, there's a cut down version of the model hosted in the cloud and a webpage that we can visit, type in a short phrase to prompt the system, and print out a few paragraphs of fake news.

I expect to suffer some degree of writer's block pretty much every day for the rest of my life. I'm a journalist and a novelist; it comes with the territory. But I have a feeling I'm going to suffer less from now on, thanks to my new best friend, GPT-2. Let me back up a bit: Six months ago, the research lab OpenAI created an AI system that generates text -- from fake news to poetry -- that in some cases actually sounds like it's written by a human being. The OpenAI team has been rolling it out in stages, each time giving us a more powerful version of the language model they dubbed GPT-2, and carefully watching to see how we use it.

Full model-generated text:"It is time once again. I believe this nation can do great things if the people make their voices heard. The men and women of America must once more summon our best elements, all our ingenuity, and find a way to turn such overwhelming tragedy into the opportunity for a greater good and the fulfillment of all our dreams. In the months and years to come, there will be many battles in which we will have to be strong and we must give all of our energy, not to repel invaders, but rather to resist aggression and to win the freedom and the equality for all of our people. The destiny of the human race hangs in the balance; we cannot afford for it to slip away. Now and in the years to come, the challenge before us is to work out how we achieve our ultimate destiny. If we fail to do so, we are doomed."

In February, an artificial intelligence lab cofounded by Elon Musk informed the world that its latest breakthrough was too risky to release to the public. OpenAI claimed it had made language software so fluent at generating text that it might be adapted to crank out fake news or spam. On Thursday, two recent master's graduates in computer science released what they say is a re-creation of OpenAI's withheld software onto the internet for anyone to download and use. Aaron Gokaslan, 23, and Vanya Cohen, 24, say they aren't out to cause havoc and don't believe such software poses much risk to society yet. The pair say their release was intended to show that you don't have to be an elite lab rich in dollars and PhDs to create this kind of software: They used an estimated $50,000 worth of free cloud computing from Google, which hands out credits to academic institutions.

The Tesla and SpaceX founder has notoriously urged for Artificial Intelligence to be respected and potentially regulated. Speaking at MIT in 2014, he called AI humanity's "biggest existential threat" and compared it to "summoning the demon". Four years on and Musk, who is usually far from a technological pessimist, reignited those fears.

Many setups in the San Francisco Bay Area boast that they are planning to change the world. However, OpenAI founded by Elon Musk has made a bigger promise than the rest: It wants to build artificial general intelligence (AGI), an AI system that like humans, can reason across many different domains and apply its skills to unfamiliar problems. For this reason, it announced a billion-dollar partnership with Microsoft to fund its work. This hints that AGI research is leaving the field of science fiction and entering the territory of serious research. "We believe that the creation of AGI will be the most important tech development in human history, with the potential to chaneg and shape the trajectory of humanity," Greg Brockman, chief technology officer (CTO) of OpenAI, informed the press.

With the advent of deep generative models in computational chemistry, in silico anticancer drug design has undergone an unprecedented transformation. While state-of-the-art deep learning approaches have shown potential in generating compounds with desired chemical properties, they entirely overlook the genetic profile and properties of the target disease. In the case of cancer, this is problematic since it is a highly genetic disease in which the biomolecular profile of target cells determines the response to therapy. Here, we introduce the first deep generative model capable of generating anticancer compounds given a target biomolecular profile. Using a reinforcement learning framework, the transcriptomic profile of cancer cells is used as a context in which anticancer molecules are generated and optimized to obtain effective compounds for the given profile. Our molecule generator combines two pretrained variational autoencoders (VAEs) and a multimodal efficacy predictor - the first VAE generates transcriptomic profiles while the second conditional VAE generates novel molecular structures conditioned on the given transcriptomic profile. The efficacy predictor is used to optimize the generated molecules through a reward determined by the predicted IC50 drug sensitivity for the generated molecule and the target profile. We demonstrate how the molecule generation can be biased towards compounds with high inhibitory effect against individual cell lines or specific cancer sites. We verify our approach by investigating candidate drugs generated against specific cancer types and investigate their structural similarity to existing compounds with known efficacy against these cancer types. We envision our approach to transform in silico anticancer drug design by increasing success rates in lead compound discovery via leveraging the biomolecular characteristics of the disease.

Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model Matthias von Davier August 21st, 2019 Abstract Objective: Showcasing Artificial Intelligence, in particular deep neural networks, for language modeling aimed at automated generation of medical education test items. Materials and Methods: OpenAI's gpt2 transformer language model was retrained using PubMed's open access text mining database. The retraining was done using toolkits based on tensorflow-gpu available on GitHub, using a workstation equipped with two GPUs. Results: In comparison to a study that used character based recurrent neural networks trained on open access items, the retrained transformer architecture allows generating higher quality text that can be used as draft input for medical education assessment material. In addition, prompted text generation can be used for production of distractors suitable for multiple choice items used in certification exams. Discussion: The current state of neural network based language models can be used to develop tools in supprt of authoring medical education exams using retrained models on the basis of corpora consisting of general medical text collections. Conclusion: Future experiments with more recent transformer models (such as Grover, TransformerXL) using existing medical certification exam item pools is expected to further improve results and facilitate the development of assessment materials. Objective The aim of this article is to provide evidence on the current state of automated item generation (AIG) using deep neural networks (DNNs). Based on earlier work, a first paper that tackled this issue used character-based Address for correspondence: mvondavier@nbme.org: Time flies in the domain of DNNs used for language modeling, indeed: The day this paper was submitted, on August 13th, 2019, to internal review, NVIDIA published yet another, larger language model of the transformer used in this paper. The MegratronLM (apart from taking a bite out of the pun in this article's title) is currently the largest language model based on the transformer architecture [3]. This latest neural network language model has 8 billions of parameters, which is incomprehensible compared to the type of neural networks we used only two decades ago. At that time, in winter semester 1999-2000, I taught classes about artificial Neural Networks (NNs, e.g. Back then, Artificial Intelligence (AI) already entered what was referred to as AI winter, as most network sizes were limited to rather small architectures unless supercomputers were employed.

In February, an artificial intelligence lab cofounded by Elon Musk informed the world that its latest breakthrough was too risky to release to the public. OpenAI claimed it had made language software so fluent at generating text that it might be adapted to crank out fake news or spam. On Thursday, two recent master's graduates in computer science released what they say is a re-creation of OpenAI's withheld software onto the internet for anyone to download and use. Aaron Gokaslan, 23, and Vanya Cohen, 24, say they aren't out to cause havoc and don't believe such software poses much risk to society yet. The pair say their release was intended to show that you don't have to be an elite lab rich in dollars and PhDs to create this kind of software: They used an estimated $50,000 worth of free cloud computing from Google, which hands out credits to academic institutions.

From Shakespeare to Austen, many writers will go down in history for their unique and engaging storytelling. But writers in the future may have competition when it comes to their stories, poems and articles - in the form of an artificial intelligence sytem. Tech firm OpenAI has developed a text generator, which many claim is now almost as good as a human writer. The system, dubbed GPT-2, was trained using a dataset of eight million web pages. This rigorous training means the AI can churn out text in different styles, ranging from Shakespeare poems to news articles.