From nuclear safety to LLM security: Applying non-probabilistic risk management strategies to build safe and secure LLM-powered systems

Large language models (LLMs) offer unprecedented and growing capabilities, but also introduce complex safety and security challenges that resist conventional risk management. While conventional probabilistic risk analysis (PRA) requires exhaustive risk enu meration and quantification, the novelty and complexity of these systems make PRA impractical, particularly against adaptive adversaries. Previous research found that risk management in various fields of engineering such as nuclear or civil engineering is often solved by generic (i.e. Here we show how emerging risks in LLM - powered systems could be met with 100+ of these non - probabilistic strategies to risk management, including risks from adaptive adversaries. The strategies are divided into five categories and are mapped to LLM secur ity (and AI safety more broadly). We also present an LLM - powered workflow for applying these strategies and other workflows suitable for solution architec ts. Overall, these strategies could contribute (despite some limitations) to security, safety and other dimensions of responsible AI.