Retrieval-augmented generation, or RAG, is a way to make a language model answer from your documents instead of from memory. When someone asks a question, the system first finds the few passages that are most likely to hold the answer, then asks the model to answer using only those passages and to show where each fact came from.
The model does not learn your documents. It reads a small, relevant slice of them each time a question is asked. That is why RAG is the usual starting point for AI search over company data.
Why not just ask a chatbot?
A general chatbot answers from what it saw during training. It has never seen your price list, your contracts or last month's policy update, so it either says it does not know or, worse, produces a confident guess. RAG fixes this by handing the model the right text at the moment of the question.
How are the documents prepared?
Before anyone can search, each document goes through a short pipeline:
- Extract the text from PDFs, Word files, spreadsheets and web pages.
- Split it into chunks of a few paragraphs. Chunks that are too large bury the answer. Chunks that are too small lose the context around it.
- Embed each chunk. An embedding model turns the text into a list of numbers that captures its meaning, so "annual leave" and "holiday allowance" end up close together.
- Store the chunks, their embeddings and their metadata, such as source file, date and who is allowed to see it.
When a document changes, only its chunks are rebuilt. Stale chunks are the most common reason a RAG system gives an out-of-date answer.
What happens when someone asks a question?
The question is embedded the same way, and the system looks for the chunks closest to it in meaning. Most good systems combine this with ordinary keyword search, because exact terms such as product codes or names are easy for keyword search and hard for meaning-based search.
A second, smaller model often re-ranks the top results so the best five or ten passages go to the language model. The model then writes the answer and cites each passage it used.
How do you keep people from seeing documents they should not?
Permissions are applied at search time, before anything reaches the model. Each chunk carries the access rules of its source document, and the search only returns chunks the person asking is allowed to read. If the filter happened after the model wrote its answer, private details could already be in the text.
How do you know the answers are right?
You test it like any other software:
- Build a question set from real questions your team asks, with the correct answer and source for each.
- Measure retrieval separately from writing. If the right passage is not in the top results, no prompt will fix the answer.
- Check citations. Every claim in an answer should point to a passage that actually says it.
- Let users flag bad answers, and add those questions to the test set.
What are the limits?
RAG is only as good as the documents behind it. If two policies contradict each other, the system may quote either one. If the answer is spread across twenty files, a single question may not pull all of it. These are content problems as much as software problems, and they are worth fixing before launch.
For a business, the practical test is simple: pick the twenty questions your team asks most often, and check whether the system answers them with the right source. That is how we scope AI search over your data.
AI and machine learning