AI Alignment and Totalitarianism

This article looks at AI misalignment through the framework of totalitarianism, as laid out in Hannah Arendt's The Origins of Totalitarianism. I don't want to make any glib moral comparisons between the very real, singular horrors of totalitarianism in the 20th century and the still hypothetical problems of AI misalignment; but I believe the parallels are worth exploring nonetheless. In her magnum opus, Arendt describes a historical and political backdrop spawning a political movement fundamentally at odds with human flourishing, such a perverse break with previous forms of government as to constitute humanity-destroying machine. Nick Bostrom's famous paper thought experiment imagines an AGI with a mandate to make as many paperclips as possible; carried out by an all-powerful agent, this banal but unconstrained (read totalitarian) reward function results in the apocalypse. Both are powerful machines that proceed logically and implacably, without the guidance natural human intuition, towards a goal fundamentally at odds with human flourishing. A totalitarian government distinguish itself from other authoritarian forms of government (even fascist dictatorships like Mussolini's Italy) in its perpetual movement towards dominating every aspect of life.