Retrieval
Answer from sources, not memory
A model knows what it saw during training. It does not know your documents, last week’s changes, or your company’s policies.
Retrieval finds relevant text when the question arrives and places it in the prompt. The model then answers from those sources. This is often called RAG: retrieval augmented generation.
1. Prepare the documents
Split documents into chunks that are small enough to be specific but large enough to make sense on their own. Keep the title, date, and link with each chunk.
2. Search when the question arrives
- Search the indexKeyword, semantic, or both
- Keep the best chunksA few relevant passages, not everything
- Answer from themUsing only the retrieved text
- Cite the sourcesSo the reader can check
3. Keyword and semantic search
Keyword search finds exact terms, such as product codes and names. Semantic search uses embeddings to find text with a similar meaning, even when the words differ.
Many systems combine both, then rerank the results to keep the most relevant ones.
4. Ground the answer
Tell the model to answer only from the provided sources and to say when they do not contain the answer. Ask it to show which passage supports each claim.
This is the same check from prompt chaining: compare the draft with the sources before showing it.
5. Most failures are search failures
If the right passage is not retrieved, the model cannot use it. When answers are wrong, look at what was retrieved before changing the prompt.
When is it useful?
When answers depend on documents that change, are private, or are too large to fit in one prompt. If the material is small and stable, putting it directly in the prompt can be simpler.