RRHF can align with not only human preferences but also any preferences. As a large language model, Wombat has the possibility to generate unsafe responses. We also conduct experiments on the IMDB dataset for assessing positive movie reviews generation. The task expects the model to give positive and fluent movie review completions based on given partial review input texts. RRHF-OP-128 follows the bottommost workflow in Figure 2 in the main texts.

Such a question is very relevant: boostinghas quickly evolvedasatechnique requiring first-order information about the loss optimized [6, Section 10.3], [41, Section 7.2.2]