Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems

Open in new window