Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems

In this work, we study rapid improvements of the training loss in transformers when being confronted with multi-step decision tasks. We found that transformers struggle to learn the intermediate task and both training and validation loss saturate for hundreds of epochs. When transformers finally learn the intermediate task, they do... Show more