But new biases are introduced due to the same LLM acting as both the evaluator and the evaluatee.

HR-based orthogonal fine-tuning is equivalent to an adaptive low-rank adaptation.

To address these challenges, we consider the two-step preference modeling procedure that first resolves the under-specification by selecting a context, and then evaluates preference with respect to the chosen context.

Moreover, our 1B model achieves exceptional performance, surpassing GPT -3.5-Turbo and Claude-3 Haiku.

We give a model of how to infer natural language rules by doing experiments.

"unsafe" queries, we posit that the scope of noncompliance should be broadened.

Object, attribute or stylistic changes can be learned from visually static datasets. On the other hand, high-quality data for action and reasoning-centric edits is scarce and has to come from entirely different sources that cover e.g.