One-Shot Safety Alignment for Large Language Models via Optimal Dualization

Ideally, we would like methods that train LMs only once ( i.e., one-shot) with a fixed objective, as in