Analysing the Generalisation and Reliability of Steering Vectors Aengus Lynch

Steering vectors (SVs) are a new approach to efficiently adjust language model behaviour at inference time by intervening on intermediate model activations. They have shown promise in terms of improving both capabilities and model alignment. However, the reliability and generalisation properties of this approach are unknown. In this work, we rigorously investigate these properties, and show that steering vectors have substantial limitations both in-and out-of-distribution. In-distribution, steerability is highly variable across different inputs.