Information, Utility & Bounded Rationality

Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we propose an axiomatic framework for bounded rational decision-making based on a thermodynamic interpretation of resource costs as information costs. We show that this axiomatic framework enforces a unique conversion law between utility and information, which can be characterized by a variational "free utility" principle akin to thermodynamical free energy. This variational principle constitutes a normative criterion that trades off utility and information costs, the latter measured by the Kullback-Leibler deviation between a distribution representing a desired policy and a reference distribution representing an initial default policy. We show that bounded optimal control solutions can be derived from this variational principle, which leads in general to stochastic policies. Furthermore, we show that risk-sensitive and robust (minimax) control schemes fall out naturally from this framework if the environment is considered as an adversarial opponent. When resource costs are ignored, the maximum expected utility principle is recovered.