Skip to content

“A token is a fundamental unit of data. So it's a word or…” — The Prof G Pod with Scott Galloway at 8:16

“A token is a fundamental unit of data. So it's a word or part of a word or a punctuation mark that is represented by a token ID. So a token ID is assigned to a bit of a word like that. And all of the large language models use these token IDs or these tokens to generate text. So every time you put a question into your…”

0:00 47:21