Skip to content
51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae episode artwork

Mar 12, 2018 16 min

51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae

from NLP Highlights Hosted by Allen Institute for Artificial Intelligence

NIPS 2017 paper by Vlad Niculae and Mathieu Blondel. Vlad comes on to tell us about his paper. Attentions are often computed in neural networks using a softmax operator, which maps scalar outputs from a model into a probability space over latent variables. There are lots of cases where this is not optimal, however, such as when you really want to encourage a sparse attention over your inputs, or when you have additional structural biases that could inform the model. Vlad and Mathieu have developed a theoretical framework for analyzing the options in this space, and in this episode we talk about that framework, some concrete instantiations of attention mechanisms from the framework, and how well these work.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae

0:00 16:36

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of NLP Highlights?

This episode is 16 minutes long.

When was this NLP Highlights episode published?

This episode was published on March 12, 2018.

Can I download this NLP Highlights episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.