RAG, or retrieval augmented generation, explained

The basic idea

A language model can generate fluent answers, but it may not know your private documents, latest files, project notes, policies, or codebase. RAG adds a retrieval step. First, the system searches a knowledge source. Then it sends the most relevant retrieved content to the model. The model uses that context to produce a grounded answer.

The word "augmented" matters. The model is not replaced by a search engine. The model is augmented with retrieved material so it can reason over information outside its training data.

Core parts of a RAG system

• Source content: documents, pages, tickets, PDFs, repos, transcripts, notes, or structured records.
• Chunking: splitting source content into retrievable pieces that preserve meaning.
• Embeddings: numerical representations that make semantic search possible.
• Index: a searchable store, often a vector database, that holds chunks and metadata.
• Retrieval: finding the chunks most relevant to a user question.
• Generation: asking the model to answer using retrieved context.
• Citations: linking the answer back to source material.
• Evaluation: testing whether retrieval and answers are correct, complete, and useful.

Why chunking matters

Chunking is one of the quiet decisions that determines RAG quality. If chunks are too small, the system retrieves fragments without enough context. If chunks are too large, retrieval can bring in irrelevant material and crowd out better evidence. Good chunking respects document structure, headings, lists, code blocks, tables, and page boundaries.

Metadata also matters. A chunk with a title, source, timestamp, owner, product area, document type, and version is easier to use responsibly than an anonymous piece of text.

RAG vs fine-tuning

RAG and fine-tuning solve different problems. RAG is usually better when the system needs access to changing knowledge, private documents, citations, or source-grounded answers. Fine-tuning is usually better for changing style, behavior, or task performance patterns. Many systems use both, but RAG is often the first choice for knowledge-base use cases.

If the question is "what does our policy say?" or "what did the architecture review conclude?", the system needs retrieval. If the question is "how should the model format every support reply?", fine-tuning may be more relevant.

Common RAG failure modes

• Bad retrieval: the right source exists, but the system does not retrieve it.
• Context overload: too many chunks dilute the important evidence.
• Stale content: old material overrides newer decisions.
• Weak citations: the answer cites sources that do not actually support the claim.
• Missing metadata: the model cannot distinguish draft notes from approved guidance.
• Answer drift: the model uses retrieved context but adds unsupported claims.

Hybrid retrieval

Many useful systems combine multiple retrieval methods. Vector search finds semantically similar text. Keyword search finds exact names, IDs, product terms, error messages, or legal phrases. Metadata filters narrow results by project, date, source type, owner, or approval status. A strong RAG system can combine these signals.

For example, a policy question may need exact language from a document, while a research question may need semantically related notes. A codebase question may need file paths and symbols. A support question may need product version and customer account metadata. Retrieval should match the shape of the domain.

RAG evaluation

RAG systems should be evaluated at both the retrieval layer and the answer layer. Retrieval evaluation asks whether the right source chunks were found. Answer evaluation asks whether the model used those chunks accurately, avoided unsupported claims, and answered the user's actual question.

Teams can create test sets from real questions. For each question, record the expected source documents, the expected answer, and unacceptable failure modes. Over time, this creates a practical evaluation harness for chunking, embeddings, filters, reranking, prompts, and model behavior.

RAG for teams

Team RAG systems need more than retrieval quality. They need access control, source freshness, ownership, review workflows, and clear provenance. A private draft should not be treated like approved policy. A stale architecture memo should not override a recent decision. A user should not retrieve documents they are not allowed to see.

That is why knowledge-base design matters. Good RAG starts before retrieval: clean sources, useful metadata, clear permissions, and a habit of preserving decisions alongside documents.

RAG in Kendr

Kendr connects RAG concepts to practical knowledge-base work. A team can build knowledge from real research, decisions, documents, and artifacts, then reuse that context in future deep research or agentic workflows.

Strong RAG is not just storage. It is the discipline of preserving source material, metadata, decisions, and evaluations so agents can answer with context and explain where the context came from.

Read Kendr Cloud Knowledge Base docs or continue with building an AI knowledge base from ongoing work.

Frequently asked RAG questions

Does RAG stop hallucinations?

RAG can reduce unsupported answers by grounding the model in retrieved sources, but it does not eliminate mistakes by itself. The system still needs good retrieval, careful prompting, citations, and evaluation.

Is vector search enough?

Sometimes, but not always. Many production systems benefit from hybrid search, metadata filters, reranking, and source-specific logic. Exact terms, IDs, dates, and permissions often matter as much as semantic similarity.

What makes a RAG answer trustworthy?

A trustworthy answer uses relevant sources, cites them accurately, marks uncertainty, avoids unsupported claims, and explains when the retrieved evidence is incomplete.

How to apply RAG thinking

Start with real user questions, not infrastructure. Write down the questions your team wants the knowledge base to answer, then identify which source collections should support those answers. That will guide chunking, metadata, permissions, and evaluation better than starting with a vector database choice.

For Kendr-style workflows, the strongest RAG use cases are research memory, project knowledge, policy review, vendor evaluation, architecture notes, and supportable decision trails.