Procedural content generation via machine learning (PCGML) has been growing in recent years. However, many PCGML approaches are only explored in the context of linear platforming games, and focused on modeling structural level information. Previously, we developed a multi-layer level representation, where each layer is designed to capture specific level information. In this paper, we apply our multi-layer approach to Lode Runner, a game with non-linear paths and complex actions. We test our approach by generating levels for Lode Runner with a constrained multi-dimensional Markov chain (MdMC) approach that ensures playability and a standard MdMC sampling approach. We compare the levels sampled when using multi-layer representation against those sampled using the single-layer representation; we compare using both the constrained sampling algorithm and the standard sampling algorithm.
Having notched impressive victories over human professionals in Go, Atari Games, and most recently StarCraft 2 -- Google's DeepMind team has now turned its formidable research efforts to soccer. In a paper released last week, the UK AI company demonstrates a novel machine learning method that trains a team of AI agents to play a simulated version of "the beautiful game." Gaming, AI and soccer fans hailed DeepMind's latest innovation on social media, with comments like "You should partner with EA Sports for a FIFA environment!" Machine learning, and particularly deep reinforcement learning, has in recent years achieved remarkable success across a wide range of competitive games. Collaborative-multi-agent games however remained a relatively difficult research domain.
Associated Press/Ahn Young-joonTV screens show the live broadcast of the Google DeepMind Challenge Match between Google's artificial intelligence program, AlphaGo, and South Korean professional Go player Lee Sedol, at the Yongsan Electronic store in Seoul, South Korea, Tuesday, March 15, 2016. Humans have been taking a beating from computers lately. The 4-1 defeat of Go grandmaster Lee Se-Dol by Google's AlphaGo artificial intelligence (AI) is only the latest in a string of pursuits in which technology has triumphed over humanity. Self-driving cars are already less accident-prone than human drivers, the TV quiz show Jeopardy! is a lost cause, and in chess humans have fallen so woefully behind computers that a recent international tournament was won by a mobile phone. There is a real sense that this month's human vs AI Go match marks a turning point.
Between 9 and 15 March 2016, a five game competition took place between Lee Sedol, the second-highest ranking professional Go player, and AlphaGo, a computer program created by Google's DeepMind subsidiary. The competition was high-stake: a prize of one million dollars was put up by Google. How exactly did AlphaGo manage to do it? All I could figure out was that machine learning was involved. Having a PhD in machine learning myself, I decided to go through the trouble and read the paper that DeepMind published on the subject. I will do my best to explain how it works in this blog post. I also read different opinions of how much a big deal this win is, and I will have some things to say about that myself (spoiler: I think it's a pretty big deal). Go and chess are very popular board games, which are similar in some respects: both are played by two players taking turns, and there is no random element involved (no dice rolling, like in backgammon). In 1997, Garry Kasparov was defeated by Deep Blue, a computer program written by IBM, running on a supercomputer. This was the first time that a reigning world chess champion was defeated by a computer program in tournament conditions.
Deep reinforcement learning has been successfully applied to several visual-input tasks using model-free methods. In this paper, we propose a model-based approach that combines learning a DNN-based transition model with Monte Carlo tree search to solve a block-placing task in Minecraft. Our learned transition model predicts the next frame and the rewards one step ahead given the last four frames of the agent's first-person-view image and the current action. Then a Monte Carlo tree search algorithm uses this model to plan the best sequence of actions for the agent to perform. On the proposed task in Minecraft, our model-based approach reaches the performance comparable to the Deep Q-Network's, but learns faster and, thus, is more training sample efficient.