Exploring Context Window of Large Language Models via Decomposed Positional Vectors

We propose two training-free context window extension methods via the lens of adjusting positional vectors, i.e., positional vector replacement and attention window extension.