Softplus Attention with Re-weighting Boosts Length Extrapolation in Large Language Models

Open in new window