Learning to Paint with Deep RL. An interesting Paper Review along with…

Teaching a machine how to paint merely using strokes is a difficult task since it needs to decide both on past history as well as for future strokes. To make the model decide based on future rewards, the authors of this paper decided to use model-based Deep Reinforcement Learning since it helps the model to learn using the future outcomes as well as using the history. They use a neural renderer to simulate the canvas for the model so that it can look into every possible outcome for a single action. Basically for each stroke on the canvas, the agent has to predict the stroke location, shape, stroke thickness and color. They propose a differentiable neural renderer that can back-propagate gradients for the model to learn.