We propose a general approach to quantitatively assessing the risk and vulnerability of artificial intelligence (AI) systems to biased decisions. The guiding principle of the proposed approach is that any AI algorithm must outperform a random guesser. This may appear trivial, but empirical results from a simplistic sequential decision-making scenario involving roulette games show that sophisticated AI-based approaches often underperform the random guesser by a significant margin. We highlight that modern recommender systems may exhibit a similar tendency to favor overly low-risk options. We argue that this "random guesser test" can serve as a useful tool for evaluating the utility of AI actions, and also points towards increasing exploration as a potential improvement to such systems.

This article looks at AI misalignment through the framework of totalitarianism, as laid out in Hannah Arendt's The Origins of Totalitarianism. I don't want to make any glib moral comparisons between the very real, singular horrors of totalitarianism in the 20th century and the still hypothetical problems of AI misalignment; but I believe the parallels are worth exploring nonetheless. In her magnum opus, Arendt describes a historical and political backdrop spawning a political movement fundamentally at odds with human flourishing, such a perverse break with previous forms of government as to constitute humanity-destroying machine. Nick Bostrom's famous paper thought experiment imagines an AGI with a mandate to make as many paperclips as possible; carried out by an all-powerful agent, this banal but unconstrained (read totalitarian) reward function results in the apocalypse. Both are powerful machines that proceed logically and implacably, without the guidance natural human intuition, towards a goal fundamentally at odds with human flourishing. A totalitarian government distinguish itself from other authoritarian forms of government (even fascist dictatorships like Mussolini's Italy) in its perpetual movement towards dominating every aspect of life.