Non-Linear Inference Time Intervention: Improving LLM Truthfulness

Secondly, the employment of an expanded information. We further developed the Inference Time token context during interventions enables a more refined construction Intervention (ITI) framework, which lets bias LLM without the of the intervention vector, thereby directing attention need for fine-tuning. The improvement manifests in introducing heads more effectively toward truthfulness. This enhanced a non-linear multi-token probing and multi-token intervention: construction of the intervention vector is attributed to the Non-Linear ITI (NL-ITI), which significantly enhances performance observation that truthful knowledge is not solely concentrated on evaluation benchmarks. NL-ITI is tested on diverse in the vector corresponding to the final token, but is distributed multiple-choice datasets, including TruthfulQA, on which we across a broader context. We discuss how our framework can report over 16 % relative MC1 (accuracy of model pointing to be used to bias LLM toward any abstract concept (truthfulness, the correct answer) improvement with respect to the baseline correctness, toxicity-prevention).