Tokenization in Natural Language Processing

from Code Logic · host Sarvesh Bhatnagar

In this episode we discuss about tokenization in Natural Language Processing. As discussed in previous episode, tokenisation is an important step in data cleaning and it entails dividing a large piece of text into smaller chunks. In this episode we discuss some of the basic tokenizers available from nltk.tokenize in nltk. If you liked this episode, do follow and do connect with me on twitter @sarvesh0829 follow my blog at www.stacklearn.org. If you sell something locally, do it using BagUp app available at play store, It would help a lot.

Episode metadata supplied by the publisher feed · Published Sep 14, 2020

Embed this episode

Attribution link and audio player

Ready to play

Tokenization in Natural Language Processing

0:00 2:12

1×

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Share this episode

Similar Episodes

No similar episodes found.

Similar Podcasts

Language Games Podcast Jon Kaus Welcome to the Language Games Podcast where we explore the true, the good, and the beautiful in philosophy, apologetics, literature, logic, and cinema (2 Cor. 10:5). The Index Podcast Index Studios The Index podcast goes beyond the code to get at the real stories behind some of today's most interesting emerging tech companies, the founders, the strategy, the stuff that actually happens behind the scenes. Every week, host Alex Kehaya sits down with entrepreneurs, builders, and investors who are shaping where the internet is headed. Alex is CEO of ABK Labs and Corbits, a platform that helps companies put AI agents into production without losing sleep over it. Corbits handles the hard parts: knowing who or what is actually acting, keeping permissions locked down, managing credentials, and keeping a clear record of what happened. Run it on Corbits' cloud or your own, with whatever model you want. We've got some news: two new co-hosts are joining the show! First up, Anne Ahola Ward from CircleClick. CNN dubbed her "The Mother of Startups," and it's a title she's earned. Anne is a technical SEO with a reputation for growing startups

Frequently Asked Questions

How long is this episode of Code Logic?

This episode is 2 minutes long.

When was this Code Logic episode published?

This episode was published on September 14, 2020.

Can I download this Code Logic episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.

URL copied to clipboard!