Skip to content
31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling episode artwork

Oct 6, 2017 11 min

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

from NLP Highlights Hosted by Allen Institute for Artificial Intelligence

ICLR 2017 paper by Hakan Inan, Khashayar Khosravi, Richard Socher, presented by Waleed. The paper presents some tricks for training better language models. It introduces a modified loss function for language modeling, where producing a word that is similar to the target word is not penalized as much as producing a word that is very different to the target (I've seen this in other places, e.g., image classification, but not in language modeling). They also give theoretical and empirical justification for tying input and output embeddings. https://www.semanticscholar.org/paper/Tying-Word-Vectors-and-Word-Classifiers-A-Loss-Fra-Inan-Khosravi/424aef7340ee618132cc3314669400e23ad910ba

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

0:00 11:19

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of NLP Highlights?

This episode is 11 minutes long.

When was this NLP Highlights episode published?

This episode was published on October 6, 2017.

Can I download this NLP Highlights episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.