What is a RAG chatbot?
A RAG chatbot is a chat interface that retrieves relevant passages from a specific document collection and answers only from those passages, rather than from the language model's general training data.
The distinction is where the answer comes from. A general chatbot answers from whatever the model absorbed in training. A RAG chatbot looks something up first — in your pages, your documents, your help centre — and answers from what it found, which is why it can cite a source.
Why it matters
It changes what a wrong answer means. A general chatbot that invents a refund policy has no traceable cause and no fix short of changing the prompt. A RAG chatbot's wrong answer traces to a specific failure: nothing relevant was retrieved, the retrieved page was out of date, or the match was thin and it answered anyway. Each of those is separately diagnosable and separately fixable.
Examples
- A support bot that answers a shipping question from the store's own delivery page and links to it.
- An internal bot that answers policy questions from an HR handbook, refusing anything the handbook does not cover.
- A documentation assistant that quotes the exact section of a reference page rather than paraphrasing from memory.
Not the same as
Fine-tuned chatbot
Fine-tuning adjusts the model's weights on your data and has to be redone when facts change. Retrieval reads current documents at answer time, so updating a page is the whole update.
Scripted chatbot
A scripted bot follows decision trees an author wrote in advance and cannot answer anything off the tree. A RAG chatbot handles unanticipated phrasing because retrieval works on meaning.
Related
tool
AI chatbot generator
Generates the configuration an AI chatbot needs — persona, tone, opening line, starter questions and the topics it can cover — derived from a real website rather than a template.
tool
Website knowledge base generator
Reads your website and organises it into a knowledge base structure — topics, the pages that belong to each, and the topics customers ask about that your site doesn't cover. Exports as Markdown.
term
What is RAG (retrieval-augmented generation)?
RAG is what separates a chatbot that knows your business from one that guesses. Documents are split into passages and indexed by meaning; each question retrieves the closest passages, and the model is instructed to answer only from them.
term
What is grounding in AI?
A grounded system retrieves passages before it writes anything, puts them in front of the model, and instructs it to answer only from what it was given. The test of whether a system is grounded is simple and does not require access to its internals: ask it something, then ask it where the answer came from. A grounded system names a page. An ungrounded one describes its reasoning.
term
What is an AI hallucination?
Language models predict plausible text, not true text. Asked a question with no supporting material, a model does not fall silent; it produces the most likely-sounding answer, which is often a wrong price, a policy that does not exist, or a citation to a page that was never written. The behaviour is not a bug to be patched out of the model. It is what the model does when nothing constrains it.
term
What is semantic search?
Keyword search asks whether two pieces of text share words. Semantic search asks whether they mean the same thing. The difference shows up immediately in customer questions, because customers do not use a company's vocabulary: someone asks 'can I send it back?' about a page headed 'Returns and exchanges', and the two share not one significant word.
RubyRep is this, applied to your website.
Give it your URL and it reads your pages, then answers your customers from them.
Train My RubyRep