Sign in

XNOR-FORMER: Learning Accurate Approximations in Long Speech Transformers

By Roshan Sharma and Bhiksha Raj
Transformers are among the state of the art for many tasks in speech, vision, and natural language processing, among others. Self-attentions, which are crucial contributors to this performance have quadratic computational complexity, which makes training on longer input sequences challenging. Prior work has produced state-of-the-art transformer variants with linear attention,... Show more
December 19, 2022
=
0
Loading PDF…
Loading full text...
Similar articles
Loading recommendations...
=
0
x1
XNOR-FORMER: Learning Accurate Approximations in Long Speech Transformers
Click on play to start listening