We propose a practical integration of logical state abstraction with AIXI, a Bayesian optimality notion for reinforcement learning agents, to significantly expand the model class that AIXI agents can be approximated over to complex history-dependent and structured environments. The state representation and reasoning framework is based on higher-order logic, which can be used to define and enumerate complex features on non-Markovian and structured environments. We address the problem of selecting the right subset of features to form state abstractions by adapting the $\Phi$-MDP optimisation criterion from state abstraction theory. Exact Bayesian model learning is then achieved using a suitable generalisation of Context Tree Weighting over abstract state sequences. The resultant architecture can be integrated with different planning algorithms.

Reinforcement Learning (RL) algorithms typically utilize learning and/or planning techniques to derive effective policies. The integration of both approaches has proven to be highly successful in addressing complex sequential decision-making challenges, as evidenced by algorithms such as AlphaZero and MuZero, which consolidate the planning process into a parametric search-policy. AIXI, the most potent theoretical universal agent, leverages planning through comprehensive search as its primary means to find an optimal policy. Here we define an alternative universal agent, which we call Self-AIXI, that on the contrary to AIXI, maximally exploits learning to obtain good policies. It does so by self-predicting its own stream of action data, which is generated, similarly to other TD(0) agents, by taking an action maximization step over the current on-policy (universal mixture-policy) Q-value estimates. We prove that Self-AIXI converges to AIXI, and inherits a series of properties like maximal Legg-Hutter intelligence and the self-optimizing property.

While mathematically elegant, a fundamental issue with AIXI is that it is only asymptotically computable.

The original AIXI reinforcement learning agent, intended as a near ly parameter-free formal gold standard for artificial general intelligence (AGI), is a Cartesian dualist that believes it is interacting with an environment from the outside, in the sense that its policy is fixed and not overwritten by anything that happens in the environment, though its actions can certainly adapt based on the percepts it receives. This is frequently compared to a person playin g a video game, who certainly does not believe he is being simulated by the game b ut rather interacts with it only by observing the screen and pressing b uttons. In contrast, it would presumably be important for an AGI to be aware t hat it exists within its environment (the universe) and its computations ar e therefore subject to the laws of physics. With this in mind, we investigate versio ns of the AIXI agent [Hut00] that treat the action sequence a on a similar footing to the percept sequence e, meaning that the actions are considered as explainable by the same rules generating the percepts. The most obvious idea is to use the universal distribution to model the joint (action/percept) dis tribution (even though actions are selected by the agent). Although this is the mos t direct way to transform AIXI into an embedded agent, it does not appear to h ave been analyzed in detail; in particular, it is usually assumed (but not proven) to fail (often implicitly, without distinguishing the universal sequence and environment distributions, e.g.

We propose a practical integration of logical state abstraction with AIXI, a Bayesian optimality notion for reinforcement learning agents, to significantly expand the model class that AIXI agents can be approximated over to complex history-dependent and structured environments. The state representation and reasoning framework is based on higher-order logic, which can be used to define and enumerate complex features on non-Markovian and structured environments. We address the problem of selecting the right subset of features to form state abstractions by adapting the \Phi -MDP optimisation criterion from state abstraction theory. Exact Bayesian model learning is then achieved using a suitable generalisation of Context Tree Weighting over abstract state sequences. The resultant architecture can be integrated with different planning algorithms.

Reinforcement Learning (RL) algorithms typically utilize learning and/or planning techniques to derive effective policies. The integration of both approaches has proven to be highly successful in addressing complex sequential decision-making challenges, as evidenced by algorithms such as AlphaZero and MuZero, which consolidate the planning process into a parametric search-policy. AIXI, the most potent theoretical universal agent, leverages planning through comprehensive search as its primary means to find an optimal policy. Here we define an alternative universal agent, which we call Self-AIXI, that on the contrary to AIXI, maximally exploits learning to obtain good policies. It does so by self-predicting its own stream of action data, which is generated, similarly to other TD(0) agents, by taking an action maximization step over the current on-policy (universal mixture-policy) Q-value estimates.

The term explanation in artificial intelligence (AI) is often conflated with the concepts of interpretability and explainable AI (XAI), but there are important distinctions to be made. Miller (2019) defines interpretability and XAI as the process of building AI systems that humans can understand. In other words, by design, the AI's decision-making process is inherently transparent to a human. In contrast, explicitly explaining the decision-making to an arbitrary human is explanation generation. The latter is the subject of this paper. More specifically, we are working towards developing a formal framework for the automated generation and assessment of explanations. Firstly, some key terminology: an explanation is generated through a dialectical interaction whereby one agent, the explainer, seeks to'explain' some phenomenon, called the explanandum, to another agent, the explainee. In this article, we propose a novel measure of explanatory effectiveness that can be used to motivate artificial agents to generate good explanations (e.g. in the form of a reward signal), or to analyse the behaviours of existing communicating agents. We then define explanation games as cooperative games where two (or more) agents seek to maximise the effectiveness measure.

Software's effect upon the world hinges upon the hardware that interprets it. This tends not to be an issue, because we standardise hardware. AI is typically conceived of as a software ``mind'' running on such interchangeable hardware. This formalises mind-body dualism, in that a software ``mind'' can be run on any number of standardised bodies. While this works well for simple applications, we argue that this approach is less than ideal for the purposes of formalising artificial general intelligence (AGI) or artificial super-intelligence (ASI). The general reinforcement learning agent AIXI is pareto optimal. However, this claim regarding AIXI's performance is highly subjective, because that performance depends upon the choice of interpreter. We examine this problem and formulate an approach based upon enactive cognition and pancomputationalism to address the issue. Weakness is a measure of plausibility, a ``proxy for intelligence'' unrelated to compression or simplicity. If hypotheses are evaluated in terms of weakness rather than length, then we are able to make objective claims regarding performance (how effectively one adapts, or ``generalises'' from limited information). Subsequently, we propose a definition of AGI which is objectively optimal given a ``vocabulary'' (body etc) in which cognition is enacted, and of ASI as that which finds the optimal vocabulary for a purpose and then constructs an AGI.

AGI already exists, it's called AIXI and it's provably optimal. It works by iterating over all programs up to a given length and choosing the shortest one that reproduces the observations. Transformers perform no iteration at all. In particular, they perform the same amount of computation for each input. This means they will never be able to reliably do reasoning or even reliably multiply two numbers together.

Artificial general intelligence (AGI) may herald our extinction, according to AI safety research. Yet claims regarding AGI must rely upon mathematical formalisms -- theoretical agents we may analyse or attempt to build. AIXI appears to be the only such formalism supported by proof that its behaviour is optimal, a consequence of its use of compression as a proxy for intelligence. Unfortunately, AIXI is incomputable and claims regarding its behaviour highly subjective. We argue that this is because AIXI formalises cognition as taking place in isolation from the environment in which goals are pursued (Cartesian dualism). We propose an alternative, supported by proof and experiment, which overcomes these problems. Integrating research from cognitive science with AI, we formalise an enactive model of learning and reasoning to address the problem of subjectivity. This allows us to formulate a different proxy for intelligence, called weakness, which addresses the problem of incomputability. We prove optimal behaviour is attained when weakness is maximised. This proof is supplemented by experimental results comparing weakness and description length (the closest analogue to compression possible without reintroducing subjectivity). Weakness outperforms description length, suggesting it is a better proxy. Furthermore we show that, if cognition is enactive, then minimisation of description length is neither necessary nor sufficient to attain optimal performance, undermining the notion that compression is closely related to intelligence. However, there remain open questions regarding the implementation of scale-able AGI. In the short term, these results may be best utilised to improve the performance of existing systems. For example, our results explain why Deepmind's Apperception Engine is able to generalise effectively, and how to replicate that performance by maximising weakness.