Skip to content
Online Learning Neural Networks: Bounds and Characterization episode artwork

May 15, 2025 13 min

Online Learning Neural Networks: Bounds and Characterization

from Neural intel Pod Hosted by Neuralintel.org

This research investigates online learning for feedforward neural networks utilizing the sign activation function. The paper identifies a margin condition in the first hidden layer as crucial for learnability, demonstrating that the optimal error bound is closely tied to the totally-separable packing number of the input space, showing an exponential dependence on dimension in some cases. To address this dimensionality issue, the authors examine two scenarios: a multi-index model where the function depends on a lower-dimensional projection, achieving better bounds, and a setting with a large margin throughout all layers, yielding bounds dependent on network depth and the number of output labels. The study also provides a method for adaptive learning when these margin parameters are unknown and extends its analysis to the agnostic learning setting.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Online Learning Neural Networks: Bounds and Characterization

0:00 13:09

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Neural intel Pod?

This episode is 13 minutes long.

When was this Neural intel Pod episode published?

This episode was published on May 15, 2025.

Can I download this Neural intel Pod episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.