What is chunking in AI retrieval?
Chunking is the process of splitting a document into smaller passages so that each can be indexed and retrieved independently.
A whole page is the wrong unit to retrieve. A returns policy page might cover eligibility, timing, refunds and exchanges; retrieving all of it to answer one question about timing buries the relevant sentence in three paragraphs of noise. Chunking splits it so that each part can be found on its own. The split usually follows paragraph or heading boundaries, because those are where a human author already decided one idea ended and another began.
Why it matters
Chunk size is the setting most likely to be blamed on the model. Chunks that are too small lose the context that made them meaningful — a line reading 'this does not apply to sale items' is useless without knowing what 'this' is. Chunks that are too large dilute the match, so a passage about four topics scores poorly on all four and loses to a page that only mentions one.
Examples
- A long FAQ page becomes one chunk per question, so a search for one question retrieves only that answer.
- A pricing page splits per plan, so 'what does the middle plan include' does not return all three.
- A blog post splits at its headings, keeping each section's context with it.
Not the same as
Tokenization
Tokenization splits text into the sub-word units a model reads. Chunking splits documents into passages a retriever can find. Different layers, different purposes.
Related
term
What is RAG (retrieval-augmented generation)?
RAG is what separates a chatbot that knows your business from one that guesses. Documents are split into passages and indexed by meaning; each question retrieves the closest passages, and the model is instructed to answer only from them.
term
What is an embedding?
Embeddings are why an AI representative can answer "do you deliver to Berlin?" from a page that only says "we ship across the EU". Text is converted to vectors, and similar meaning lands close together.
term
What is semantic search?
Keyword search asks whether two pieces of text share words. Semantic search asks whether they mean the same thing. The difference shows up immediately in customer questions, because customers do not use a company's vocabulary: someone asks 'can I send it back?' about a page headed 'Returns and exchanges', and the two share not one significant word.
term
What is a token in AI?
Models do not read characters or words; they read tokens. For ordinary English text the working ratio is about four characters per token, or roughly seven hundred and fifty words per thousand tokens. Common words are usually one token each, unusual ones split into several, and a long product code may become five or six.
term
What is grounding in AI?
A grounded system retrieves passages before it writes anything, puts them in front of the model, and instructs it to answer only from what it was given. The test of whether a system is grounded is simple and does not require access to its internals: ask it something, then ask it where the answer came from. A grounded system names a page. An ungrounded one describes its reasoning.
term
What is a token in AI?
Models do not read characters or words; they read tokens. For ordinary English text the working ratio is about four characters per token, or roughly seven hundred and fifty words per thousand tokens. Common words are usually one token each, unusual ones split into several, and a long product code may become five or six.
RubyRep is this, applied to your website.
Give it your URL and it reads your pages, then answers your customers from them.
Train My RubyRep