The current success of RL highly relies on excessive amount ofdata, which, however,isusually not available inmanyreal world tasks wheredeploying anew policyisverycostlyorevenrisky.

Work performed while at Stanford University.

Releasing open-source large language models (LLMs) presents a dual-use risk since bad actors can easily fine-tune these models for harmful purposes. Even without the open release of weights, weight stealing and fine-tuning APIs make closed models vulnerable to harmful fine-tuning attacks (HFAs).

Deep Neural Networks are vulnerable toTrojan (or backdoor) attacks.