What is a context window?
A context window is the maximum amount of text, measured in tokens, that a language model can take into account when producing a single response.
Everything a model considers — the instructions, the retrieved passages, the conversation so far and the answer being written — has to fit inside the context window. Windows have grown enormously, to the point where a modest website could be pasted into one wholesale, which raises the obvious question of why anyone still retrieves.
Why it matters
Two reasons the size alone does not settle it. The first is cost: you pay per token on every question, so sending an entire site to answer 'what are your opening hours' means paying for the whole site, every time, forever. The second is accuracy — models attend less reliably to material buried in the middle of a very long input, so drowning one relevant paragraph in two hundred irrelevant ones makes the answer worse, not better. Retrieval is what keeps the window full of the right text.
Examples
- A question, three retrieved passages and the last few turns of conversation, well within any modern window.
- A long conversation that has to be trimmed, dropping the earliest turns to make room.
- A whole website pasted in, which fits, costs a hundred times more, and answers slightly worse.
Not the same as
Memory
A context window is per request and disappears afterwards. Anything a system remembers between conversations is stored somewhere and re-inserted, not retained by the model.
Related
term
What is a token in AI?
Models do not read characters or words; they read tokens. For ordinary English text the working ratio is about four characters per token, or roughly seven hundred and fifty words per thousand tokens. Common words are usually one token each, unusual ones split into several, and a long product code may become five or six.
term
What is RAG (retrieval-augmented generation)?
RAG is what separates a chatbot that knows your business from one that guesses. Documents are split into passages and indexed by meaning; each question retrieves the closest passages, and the model is instructed to answer only from them.
term
What is a large language model?
The training objective is narrow — predict the next piece of text — and the capabilities that emerge from it are broad. A model that has read enough text learns grammar, tone, structure, and a great deal of general world knowledge, without ever being taught any of those things directly.
term
What is a large language model?
The training objective is narrow — predict the next piece of text — and the capabilities that emerge from it are broad. A model that has read enough text learns grammar, tone, structure, and a great deal of general world knowledge, without ever being taught any of those things directly.
term
What is a token in AI?
Models do not read characters or words; they read tokens. For ordinary English text the working ratio is about four characters per token, or roughly seven hundred and fifty words per thousand tokens. Common words are usually one token each, unusual ones split into several, and a long product code may become five or six.
term
What is conversational AI?
The term is used loosely enough to cover almost anything with a text box, but there is a practical test, and it is a follow-up question. Ask 'do you ship to Ireland?', get an answer, then ask 'how much?'. A system that answers the cost of shipping to Ireland is conversational. One that asks 'how much of what?' — or worse, answers about product pricing — is a form with a chat interface.
RubyRep is this, applied to your website.
Give it your URL and it reads your pages, then answers your customers from them.
Train My RubyRep