Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization

Models (LLMs) and build personalized LLMs tailored for various applications. While fine-tuning seems to be a direct solution, it requires substantial computational resources and may significantly affect the utility of the original LLM.