Figure crepanc bythe which thee circles which differ .

A fundamental question in neuroscience is how the brain creates an internal model of the world to guide actions using sequences of ambiguous sensory information. This is naturally formulated as a reinforcement learning problem under partial observations, where an agent must estimate relevant latent variables in the world from its evidence, anticipate possible future states, and choose actions that optimize total expected reward. This problem can be solved by control theory, which allows us to find the optimal actions for a given system dynamics and objective function. However, animals often appear to behave suboptimally.

Weaknesses: The specific empirical evaluation chosen is the primary weakness of the paper. From a neuroscience perspective, the validation of parameter recovery on synthetic data is a necessary first step, but not a sufficient one. Given that [a] the task is primarily of neuroscientific interest and [b] a simpler (though also bayesian belief-updating) fit model is given in the cited prior work, the lack of comparison of cross-validated performance against that prior model is surprising. We should either see better cross-validation performance to the models in prior work, or similar performance but more insight / explanation of the underlying mental computation. This would show us a real payoff of the new insights here.

The paper describes a novel technique for inverse rational control. The reviewers all agree that this is great work that makes an important contribution. There is one important weakness though: the experiments. More comprehensive experiments would be desirable to increase the impact of the work. Nevertheless, this is still good work.

A fundamental question in neuroscience is how the brain creates an internal model of the world to guide actions using sequences of ambiguous sensory information. This is naturally formulated as a reinforcement learning problem under partial observations, where an agent must estimate relevant latent variables in the world from its evidence, anticipate possible future states, and choose actions that optimize total expected reward. This problem can be solved by control theory, which allows us to find the optimal actions for a given system dynamics and objective function. However, animals often appear to behave suboptimally. We hypothesize that animals have their own flawed internal model of the world, and choose actions with the highest expected subjective reward according to that flawed model.