Sign in

Efficient Contextual Representation Learning Without Softmax Layer

By Liunian Harold Li and others
Contextual representation models have achieved great success in improving various downstream tasks. However, these language-model-based encoders are difficult to train due to the large parameter sizes and high computational complexity. By carefully examining the training procedure, we find that the softmax layer (the output layer) causes significant inefficiency due to... Show more
February 28, 2019
=
0
Loading PDF…
Loading full text...
Similar articles
Loading recommendations...
=
0
x1
Efficient Contextual Representation Learning Without Softmax Layer
Click on play to start listening