“A token is a fundamental unit of data. So it's a word or…” — The Prof G Pod with Scott Galloway at 8:16
“A token is a fundamental unit of data. So it's a word or part of a word or a punctuation mark that is represented by a token ID. So a token ID is assigned to a bit of a word like that. And all of the large language models use these token IDs or these tokens to generate text. So every time you put a question into your…”
0:00
47:21