Recursive Backwards Q-Learning in Deterministic Environments

Machine learning and reinforcement learning are increasingly popular and important fields in the modern age. There are problems that reinforcement learning agents can learn to solve more efficiently and consistently than any human when given enough time to practice. However, modern approaches like Q-learning run into issues when facing certain types of problems. Their approach to solving problems in combination with not using a model of the environment causes them to take longer than is necessary to learn to solve problems that are deterministic in nature. By working without model of the environment, information that is available and help the learning process is ignored. This paper introduces an adapted Q-learning agent called the recursive backwards Q-Learning (RBQL) agent. It solves these types of problems by building a model of its environment as it explores and recursively applying the Q-value update rule to find an optimal policy much quicker than a regular Q-learning agent. This agent is shown to work with the example of finding the fastest path through a maze. Its results are compared to the results of a regular Q-learning agent.