In this paper, we focus on learning intelligent agents through model-free reinforcement learning. Rather than arguing that reinforcement learning is the right abstraction for attaining intelligent behavior, we consider the issue of finding useful abstractions to represent the agent and the environment when verification is in order. Indeed, verifying that the agent's behavior complies to some stated safety property -- an "Asimovian" perspective -- only adds to the challenge that abstracting intelligence represents per se. In the paper, we show an example application about verification of abstractions in model-free learning, and we argue about potential (more) useful abstractions in the same context.

The best way to compare two frameworks is to code something up in both of them. I've written a companion jupyter notebook for this post and you can get it here. All code will be provided in the post too. First, let's code a simple approximator for the following function in both frameworks: We will try to find unknown parameter phi given data x and function values f(x). Yes, using stochastic gradient descent for this is an overkill and analytical solution may be found easily, but this problem will serve our purpose well as a simple example.