Estimating catastrophic harms from frontier AI is hindered by deep ambiguity: many of its risks are not only unobserved but unanticipated by analysts. The central limitation of current risk analysis is the inability to populate the $\textit{catastrophic event space}$, or the set of potential large-scale harms to which probabilities might be assigned. This intractability is worsened by the $\textit{Lucretius problem}$, or the tendency to infer future risks only from past experience. We propose a process of $\textit{dark speculation}$, in which systematically generating and refining catastrophic scenarios ("qualitative" work) is coupled with estimating their likelihoods and associated damages (quantitative underwriting analysis). The idea is neither to predict the future nor to enable insurance for its own sake, but to use narrative and underwriting tools together to generate probability distributions over outcomes. We formalize this process using a simplified catastrophic Lévy stochastic framework and propose an iterative institutional design in which (1) speculation (including scenario planning) generates detailed catastrophic event narratives, (2) insurance underwriters assign probabilistic and financial parameters to these narratives, and (3) decision-makers synthesize the results into summary statistics to inform judgment. Analysis of the model reveals the value of (a) maintaining independence between speculation and underwriting, (b) analyzing multiple risk categories in parallel, and (c) generating "thick" catastrophic narrative rich in causal (counterfactual) and mitigative detail. While the approach cannot eliminate deep ambiguity, it offers a systematic approach to reason about extreme, low-probability events in frontier AI, tempering complacency and overreaction. The framework is adaptable for iterative use and can be further augmented with AI systems.

When the board of OpenAI staged a bum mutiny last November, throwing out the company's leadership only to have the bosses return while board members were pressured to resign, something seemed rotten in the state of effective altruism. Nominally, OpenAI's mission had been to ensure that AI "benefits all of humanity." Fiduciarily, OpenAI's mission is to benefit the subset of humanity with a stake in OpenAI. And then, of course, there was Sam Bankman-Fried, the felonious altruist who argued in court last fall that his sordid crypto exchange was in fact a noble exercise in earning-to-give--making Midas money, sure, but only to funnel it to the global poor. This week he's facing a prison sentence of up to 50 years, which his legal team has complained paints him as a "depraved super-villain."

Increasingly, we're surrounded by fake people. Sometimes we know it and sometimes we don't. They offer us customer service on Web sites, target us in video games, and fill our social-media feeds; they trade stocks and, with the help of systems such as OpenAI's ChatGPT, can write essays, articles, and e-mails. By no means are these A.I. systems up to all the tasks expected of a full-fledged person. But they excel in certain domains, and they're branching out. Many researchers involved in A.I. believe that today's fake people are just the beginning.

On Tuesday, OpenAI announced the release of GPT-4, its latest, biggest language model, only a few months after the splashy release of ChatGPT. GPT-4 was already in action -- Microsoft has been using it to power Bing's new assistant function. The people behind OpenAI have written that they think the best way to handle powerful AI systems is to develop and release them as quickly as possible, and that's certainly what they're doing. Also on Tuesday, I sat down with Holden Karnofsky, the co-founder and co-CEO of Open Philanthropy, to talk about AI and where it's taking us. Karnofsky, in my view, should get a lot of credit for his prescient views on AI.

In each of the scenarios below, I'll discuss specific impacts we can expect from that scenario. In these impact sections, I'll discuss general impacts on the four components of alignment presented above. I also consider more in depth how each of these scenarios impacts several specific robustness and alignment techniques. To help keep the main text of this post from becoming too lengthy, I have placed this analysis in Appendix 1: Analysis of scenario impacts on specific robustness and alignment techniques. I link to the relevant parts of this appendix analysis throughout the main scenarios analysis below.

We don't have much reason to think that they have an internal monologue, the kind of sense perception humans have, or an awareness that they're a being in the world. Over the weekend, the Washington Post's Nitasha Tiku published a profile of Blake Lemoine, a software engineer assigned to work on the Language Model for Dialogue Applications (LaMDA) project at Google. LaMDA is a chatbot AI, and an example of what machine learning researchers call a "large language model," or even a "foundation model." It's similar to OpenAI's famous GPT-3 system, and has been trained on literally trillions of words compiled from online posts to recognize and reproduce patterns in human language. LaMDA is a really good large language model.