While there is universal agreement that agents ought to act ethically, there is no agreement as to what constitutes ethical behaviour. To address this problem, recent philosophical approaches to `moral uncertainty' propose aggregation of multiple ethical theories to guide agent behaviour. However, one of the foundational proposals for aggregation - Maximising Expected Choiceworthiness (MEC) - has been criticised as being vulnerable to fanaticism; the problem of an ethical theory dominating agent behaviour despite low credence (confidence) in said theory. Fanaticism thus undermines the `democratic' motivation for accommodating multiple ethical perspectives. The problem of fanaticism has not yet been mathematically defined. Representing moral uncertainty as an instance of social welfare aggregation, this paper contributes to the field of moral uncertainty by 1) formalising the problem of fanaticism as a property of social welfare functionals and 2) providing non-fanatical alternatives to MEC, i.e. Highest k-trimmed Mean and Highest Median.

First, we need to clear up terminology, like A.I. Safety, which sounds like a completely neutral, uncontroversial term. Who wouldn't want safe A.I. software? And you might think that the definition of A.I. "safety" would include A.I. that isn't racist or sexist or is used to abet genocide. All of which, by the way, are actual, documented concerns about today's existing A.I. software. Yet actually, none of those concerns are what A.I. researchers generally mean when they talk about "A.I. Instead, those things fall into the camp of "A.I.

Even during an actual pandemic, Flynn's focus struck many Oregonians as far-fetched and foreign. Perhaps unsurprisingly, he ended up losing the 2022 primary to the more politically experienced Democrat, Andrea Salinas. But despite Flynn's lackluster showing, he made history as effective altruism's first political candidate to run for office. Since its birth in the late 2000s, effective altruism has aimed to answer the question "How can those with means have the most impact on the world in a quantifiable way?"--and supplied clear methodologies for calculating the answer. Directing money to organizations that use evidence-based approaches is the one technique EA is most known for.

Recently the philosopher William MacAskill, with his book What We Owe The Future, has been popularizing the idea that the fate of humanity should be our top moral priority. His core proposition is that today's 8 billion humans are vastly outweighed in importance by the hundreds of billions of humans who could live in future generations if we can avoid wiping out humanity in the near term. MacAskill's argument is known by the slogan "longtermism," (often written as long-termism) and it has already been sharply criticized. For example, columnist Christine Emba has written in The Washington Post: "It's compelling at first blush, but as a value system, its practical implications are worrisome." In practice, she explains, it implies seeing "preventing existential threats to humanity as the most valuable philanthropic cause"--which means we should invest far more in addressing risks that threaten humanity's very long-term existence. As Emba says, this can seem impossible to disagree with.

Humanity is a wayward teenager. Our species has its whole life ahead of it, but the decisions we make now will irrevocably shape the course of our adulthood. We could recognize the stakes of this critical moment, buckle down, do our homework, drink responsibly, eat sustainably, prepare for pandemics, avert robot apocalypses, realize our full potential, and live a long, prosperous, meaningful life before dying peacefully in a supernova at the ripe old age of 1 trillion. Or we could party all the time, get into fights, start a nuclear war, create doomsday bioweapons, tremble before our new robot overlords, live fast, die young, and leave an irradiated corpse. We owe it to our future selves -- which is to say, to the hundreds of billions of potential future humans -- to choose wisely.

Although most cultures, particularly in the west, provide a great many commemorations of distant ancestors – statues, portraits, buildings – we are much less willing to consider our far-off descendants. We might invoke grandchildren, at a push great-grandchildren, but after that, it all becomes a bit vague and, well, unimaginable. And while we look with awe and fascination at the Egyptian pyramids, built 5,000 years ago, we seem incapable of thinking, or even contemplating, 5,000 years in the future. That lies in the realm of science fiction, which is tantamount to fantasy. But the chances are, barring a global catastrophe, humanity will still be very much around in 5,000 years, and going by the average existence of mammal species, should still be thriving in 500,000 years. If we play our cards right, we could even be here in 5m or 500m years, which means that there may be thousands or even millions times more human beings to come than have already existed.