This position paper contends that modern AI research must adopt an antifragile perspective on safety -- one in which the system's capacity to guarantee long-term AI safety such as handling rare or out-of-distribution (OOD) events expands over time. Conventional static benchmarks and single-shot robustness tests overlook the reality that environments evolve and that models, if left unchallenged, can drift into maladaptation (e.g., reward hacking, over-optimization, or atrophy of broader capabilities). We argue that an antifragile approach -- Rather than striving to rapidly reduce current uncertainties, the emphasis is on leveraging those uncertainties to better prepare for potentially greater, more unpredictable uncertainties in the future -- is pivotal for the long-term reliability of open-ended ML systems. In this position paper, we first identify key limitations of static testing, including scenario diversity, reward hacking, and over-alignment. We then explore the potential of antifragile solutions to manage rare events. Crucially, we advocate for a fundamental recalibration of the methods used to measure, benchmark, and continually improve AI safety over the long term, complementing existing robustness approaches by providing ethical and practical guidelines towards fostering an antifragile AI safety community.

With this paper, we aim to put an issue on the agenda of AI ethics that in our view is overlooked in the current discourse. The current discussions are dominated by topics suchas trustworthiness and bias, whereas the issue we like to focuson is counter to the debate on trustworthiness. We fear that the overuse of currently dominant AI systems that are driven by short-term objectives and optimized for avoiding error leads to a society that loses its diversity and flexibility needed for true progress. We couch our concerns in the discourse around the term anti-fragility and show with some examples what threats current methods used for decision making pose for society.

Contrary to what some data scientists may like to believe, we can never reduce the world to mere numbers and algorithms. When it comes down to it, decisions are made by humans, and being an effective data scientist means understanding both people and data. When OPower, a software company, wanted to get people to use less energy, they provided customers with plenty of stats about their electricity usage and cost. However, the data alone were not enough to get people to change. In addition, OPower needed to take advantage of behavioral science, namely, studies showing people were driven to reduce energy when they received smiley emoticons on their bills showing how they compare to their neighbors!

It began, late last year, with Silver boasting about the success of his election models and Taleb shooting back that Silver doesn't "know how math works." Silver said Taleb was "consumed by anger" and hadn't had any new ideas since 2001. The argument has gotten personal, with Silver calling Taleb an "intellectual-yet-idiot" (an insult taken from Taleb's own book) and Taleb calling Silver "klueless" and "butthurt." Here is a recap of what they're fighting about so you can know who's right (Silver, mostly) and who's wrong (Taleb). The origin of Taleb's ire can be found in Silver's success since 2008--and his some-time failures. As I described in Nautilus last month, evaluating probabilistic election forecasts can be conceptually slippery, made especially difficult by the counterintuitive properties of mathematical probability.