What is semantic search?
Semantic search finds text by meaning rather than by shared words, using numeric representations of passages so that similar ideas sit close together.
Keyword search asks whether two pieces of text share words. Semantic search asks whether they mean the same thing. The difference shows up immediately in customer questions, because customers do not use a company's vocabulary: someone asks 'can I send it back?' about a page headed 'Returns and exchanges', and the two share not one significant word.
Why it matters
Every business writes its website in its own words, and every visitor asks in theirs. Keyword search puts the burden of translation on the customer, which is why site search so often returns nothing and the visitor leaves. Semantic search removes that burden, which is most of the reason an AI representative outperforms the search box that was already on the page.
Examples
- 'How long till it arrives?' matches a page titled 'Delivery estimates'.
- 'Do you take Amex?' matches 'Accepted payment methods'.
- 'Is there a way to cancel?' matches 'Ending your subscription'.
Not the same as
Keyword search
Keyword search matches literal terms and is faster and more predictable. The two are often combined, with keywords as a fallback when semantic scores are weak.
Vector database
A vector database is one place to store the numbers semantic search compares. The technique does not require one.
Related
term
What is an embedding?
Embeddings are why an AI representative can answer "do you deliver to Berlin?" from a page that only says "we ship across the EU". Text is converted to vectors, and similar meaning lands close together.
term
What is cosine similarity?
It is the standard way to compare embeddings, and its defining property is that it ignores magnitude entirely. Two vectors pointing the same way score identically whether one is twice as long as the other. In practice, scores for text embeddings tend to fall in a narrow band — often 0.6 to 0.9 for anything vaguely related — which is why a raw score is meaningless without knowing the distribution for a given model.
term
What is RAG (retrieval-augmented generation)?
RAG is what separates a chatbot that knows your business from one that guesses. Documents are split into passages and indexed by meaning; each question retrieves the closest passages, and the model is instructed to answer only from them.
term
What is a vector database?
Comparing a question to every stored passage is exact and gets slower in proportion to how many passages there are. A vector database trades a little accuracy for a large speed gain by building an index that skips most of the comparisons. That trade is worth making at millions of vectors and is close to pointless at thousands.
tool
Website support score
Scores a website on how well it could support customers — and how well an AI representative trained on it would perform. Five dimensions, a total out of 100, and the specific gaps behind the score.
use case
Self-service support that finds the answer
When self-service fails, the instinct is to write more articles. Often the article already exists and the visitor could not find it, because help centre search matches words rather than meaning and customers rarely use the words the article was written in.
RubyRep is this, applied to your website.
Give it your URL and it reads your pages, then answers your customers from them.
Train My RubyRep