Gwern has answered my prayers and taught GPT-2 poetry. GPT-2 is the language processing system that OpenAI announced a few weeks ago. They are keeping the full version secret, but have released a smaller prototype version. Gwern retrained it on the Gutenberg Poetry Corpus, a 117 MB collection of pre-1923 English poetry, to create a specialized poetry AI. I previously tested the out-of-the-box version of GPT-2 and couldn't make it understand rhyme and meter.

Artificial intelligence (AI) and machine learning architectures such as deep learning have become integral parts of our daily lives--they enable digital speech assistants or translation services, improve medical diagnostics and are an indispensable part of future technologies such as autonomous driving. Based on an ever increasing amount of data and powerful novel computer architectures, learning algorithms seemingly approach human capabilities, sometimes even surpassing them. So far, however, it often remains unknown to users how exactly AI systems reach their conclusions. Therefore, it may often remain unclear whether the AI's decision-making behavior is truly intelligent or whether the procedures are just averagely successful. Researchers from TU Berlin, Fraunhofer Heinrich Hertz Institute HHI and Singapore University of Technology and Design (SUTD) have tackled this question and have provided a glimpse into the diverse "intelligence" spectrum observed in current AI systems, specifically analyzing these AI systems with a novel technology that allows automatized analysis and quantification.

It's no secret to marketers that effective use of data is critical. It is the key to executing successful campaigns that engage consumers and drive towards long-term, profitable relationships, and artificial intelligence (AI) and machine learning are vital parts of analysing and optimizing data at scale. The amount of available data on consumers and their habits, preferences and behaviours continues to grow. It is therefore increasingly challenging to make sense of the data and make accurate predictions. Consumers sharing information about themselves via social media and e-commerce using mobile devices and personal computers are leaving behind data that can prove extremely valuable if marketers are able to look at all the data points to build holistic profiles of past, current and prospective customers.

Just two years later, months after DeepMind Health had closed contracts with partner hospitals, Google announced Google Health and absorbed DeepMind's program into it, giving little notice to DeepMind or the hospitals. Sources close to the matter, granted anonymity by 1843 because they signed non-disclosure agreements, shared that the move led to resentment and anger among DeepMind's team. Some of those same anonymous sources also shared doubts with 1843 that DeepMind would ever crack the code of artificial general intelligence, pointing to little-known weaknesses and caveats to the company's highly-publicized AI success stories. The algorithms developed at DeepMind can do fascinating things, like demolish certain video games and retain knowledge better than other systems, but there's no clear roadmap to the fundamentally-new technology necessary for developing AGI. But should Hassabis ever figure it out, the safeguards and ethical review boards that he put in place when his company was bought could mean that the technology will remain in the hands of him and his team instead of becoming just another Google technology.

In just 30 years' time, it is forecasted that the human population of our planet will be close to 10 billion. Producing enough food to feed these hungry mouths will be a challenge, and demographic trends such as urbanization, particularly in developing countries, will only add to that. To meet that challenge, agricultural businesses are pinning their hopes on technology, and that idea that increasingly sophisticated data and analytics tools will help to drive efficiencies and cut waste in agriculture and food production. Leading the way is John Deere – the 180-year-old manufacturer of farming and industrial machinery which has spent the past decade transforming itself into an artificial intelligence (AI) and data-driven business. I have covered John Deere before here.

GPT-2 is Open AI's language-generation model (last seen around these parts as a means of detecting machine-generated text); it's powerful and cool, and Gwern Branwen fed it the Project Gutenberg poetry corpus to see what kind of poetry it would write. Pretty good poetry, as it turns out. Scott Alexander (previously) does a good job of highlighting its greatest hits (and misses). GPT-2 poetry manages some impressive accomplishments, like solid iambic pentameter ("Thou know'st how Menoetiades the swift/Was dragged, of Hector and the fierce compeers/And Phrygian warriors. So, we will dispatch/Your bodies, then, yourselves to burn the ships/In sacrifice; with torches and with bells") though it struggles with extended/complex rhyming ("If sick of sense; Wholubil, or snug.

In model-based reinforcement learning, the agent interleaves between model learning and planning. These two components are inextricably intertwined. If the model is not able to provide sensible long-term prediction, the executed planner would exploit model flaws, which can yield catastrophic failures. This paper focuses on building a model that reasons about the long-term future and demonstrates how to use this for efficient planning and exploration. To this end, we build a latent-variable autoregressive model by leveraging recent ideas in variational inference. We argue that forcing latent variables to carry future information through an auxiliary task substantially improves long-term predictions. Moreover, by planning in the latent space, the planner's solution is ensured to be within regions where the model is valid. An exploration strategy can be devised by searching for unlikely trajectories under the model. Our method achieves higher reward faster compared to baselines on a variety of tasks and environments in both the imitation learning and model-based reinforcement learning settings.

From the beginning if the history of AI, there has been interest in games as a platform of research. As the field developed, human-level competence in complex games became a target researchers worked to reach. Only relatively recently has this target been finally met for traditional tabletop games such as Backgammon, Chess and Go. Current research focus has shifted to electronic games, which provide unique challenges. As is often the case with AI research, these results are liable to be exaggerated or misrepresented by either authors or third parties. The extent to which these games benchmark consist of fair competition between human and AI is also a matter of debate. In this work, we review the statements made by authors and third parties in the general media and academic circle about these game benchmark results and discuss factors that can impact the perception of fairness in the contest between humans and machines

In traditional generative modeling, good data representation is very often a base for a good machine learning model. It can be linked to good representations encoding more explanatory factors that are hidden in the original data. With the invention of Generative Adversarial Networks (GANs), a subclass of generative models that are able to learn representations in an unsupervised and semi-supervised fashion, we are now able to adversarially learn good mappings from a simple prior distribution to a target data distribution. This paper presents an overview of recent developments in GANs with a focus on learning latent space representations.

We extend the existing framework of semi-implicit variational inference (SIVI) and introduce doubly semi-implicit variational inference (DSIVI), a way to perform variational inference and learning when both the approximate posterior and the prior distribution are semi-implicit. In other words, DSIVI performs inference in models where the prior and the posterior can be expressed as an intractable infinite mixture of some analytic density with a highly flexible implicit mixing distribution. We provide a sandwich bound on the evidence lower bound (ELBO) objective that can be made arbitrarily tight. Unlike discriminator-based and kernel-based approaches to implicit variational inference, DSIVI optimizes a proper lower bound on ELBO that is asymptotically exact. We evaluate DSIVI on a set of problems that benefit from implicit priors. In particular, we show that DSIVI gives rise to a simple modification of VampPrior, the current state-of-the-art prior for variational autoencoders, which improves its performance.