What is a token in AI?
A token is the unit of text a language model processes — roughly a word or part of a word, produced by splitting text into common sub-word pieces.
Models do not read characters or words; they read tokens. For ordinary English text the working ratio is about four characters per token, or roughly seven hundred and fifty words per thousand tokens. Common words are usually one token each, unusual ones split into several, and a long product code may become five or six.
Why it matters
Tokens are the billing unit, and they are counted twice: once for everything sent to the model and once for everything it writes back. That is why retrieval quality shows up directly as cost. A system that sends five well-chosen passages per question pays for five passages; one that sends twenty in the hope that the right one is among them pays four times as much for a worse answer, on every question, forever. Tokens are also what the context window is measured in, so they cap how much can be considered at once.
Examples
- A typical support question, about fifteen tokens.
- Three retrieved passages, roughly six hundred.
- A two-paragraph answer, about a hundred and twenty.
Not the same as
Word
Tokens do not align with words. 'Unbelievable' may be three tokens; 'the' is one.
Chunk
A chunk is a passage a retriever finds. A chunk is made of tokens, but the two are chosen for different reasons.
Related
term
What is a context window?
Everything a model considers — the instructions, the retrieved passages, the conversation so far and the answer being written — has to fit inside the context window. Windows have grown enormously, to the point where a modest website could be pasted into one wholesale, which raises the obvious question of why anyone still retrieves.
term
What is chunking in AI retrieval?
A whole page is the wrong unit to retrieve. A returns policy page might cover eligibility, timing, refunds and exchanges; retrieving all of it to answer one question about timing buries the relevant sentence in three paragraphs of noise. Chunking splits it so that each part can be found on its own. The split usually follows paragraph or heading boundaries, because those are where a human author already decided one idea ended and another began.
term
What is a large language model?
The training objective is narrow — predict the next piece of text — and the capabilities that emerge from it are broad. A model that has read enough text learns grammar, tone, structure, and a great deal of general world knowledge, without ever being taught any of those things directly.
tool
Chatbot ROI calculator
Estimates the support cost an AI chatbot would remove, based on your ticket volume, handling time and loaded hourly cost — plus payback period against a monthly subscription.
term
What is chunking in AI retrieval?
A whole page is the wrong unit to retrieve. A returns policy page might cover eligibility, timing, refunds and exchanges; retrieving all of it to answer one question about timing buries the relevant sentence in three paragraphs of noise. Chunking splits it so that each part can be found on its own. The split usually follows paragraph or heading boundaries, because those are where a human author already decided one idea ended and another began.
term
What is a context window?
Everything a model considers — the instructions, the retrieved passages, the conversation so far and the answer being written — has to fit inside the context window. Windows have grown enormously, to the point where a modest website could be pasted into one wholesale, which raises the obvious question of why anyone still retrieves.
RubyRep is this, applied to your website.
Give it your URL and it reads your pages, then answers your customers from them.
Train My RubyRep