What is Retrieval Augmented Generation (RAG)?

Large language models are impressive, but they have two big blind spots: they only know what was in their training data, and they sometimes make things up. Retrieval-augmented generation (RAG) fixes much of that by letting an AI look up relevant information before it answers.

This guide explains what RAG is, why language models need it, how it works step by step, how it compares with fine-tuning, the main types, real-world examples, its limits, and how to get started.

Retrieval-augmented generation is a technique that combines a search step with a large language model. When you ask a question, a RAG system first retrieves relevant information from an external source, such as company documents, a database, or the web, then gives that information to the model so it can generate an answer grounded in real sources.

Think of it as the difference between a closed-book and an open-book exam. A standard model answers from memory alone, while a RAG system gets to check its notes first. Ask a company assistant "How many vacation days do new employees get?" and a RAG system will find the relevant section of the HR handbook and answer based on that page, often with a link to it, instead of guessing from general knowledge.

Even the most capable generative AI models share three limitations. Their knowledge stops at a training cutoff, so they miss recent events. They have never seen your private data, such as internal policies, product manuals, or customer records. And when they do not know something, they may produce a confident but wrong answer, known as a hallucination. RAG addresses all three by supplying fresh, specific, verifiable information at the moment of each question, without retraining the model.

A typical RAG system works in four stages. Here is how they apply to the HR handbook example:

The key idea is semantic search. Embeddings, produced by machine learning models, capture meaning rather than exact wording, so a question about "time off" can still find a section titled "paid leave." Vector databases such as Pinecone, Weaviate, and Chroma are built to run these similarity searches quickly across millions of chunks.

RAG is not the only way to give a model more knowledge. The two main alternatives are fine-tuning the model and simply pasting more text into its context window.

In practice, many systems combine approaches, for example a fine-tuned model that also uses RAG for up-to-date facts.

RAG has evolved from a simple pipeline into more sophisticated designs.

Agentic RAG is the idea behind research modes such as Deep Research, which plan a series of searches before writing a report. Standards like the Model Context Protocol make it easier for these agents to connect to many different data sources.

If you have used an AI assistant that searches the web and cites its sources, you have already used RAG. Tools like ChatGPT with search, and other AI apps with built-in web search, retrieve current pages and ground their answers in them. The same principle powers features that answer questions about a document you upload, much like a PDF summarizer that reads a file before summarizing it.

Businesses use RAG to build internal assistants that answer employee questions from company wikis and policies, and customer support AI chatbots that pull answers from product manuals and help centers. Law firms, financial analysts, and researchers use it to search large document collections, and healthcare teams use it to surface relevant guidelines and literature, with appropriate privacy safeguards.

RAG is only as good as what it retrieves. If the search step misses the relevant document, or pulls in outdated or low-quality sources, the answer will suffer no matter how capable the model is. Poorly chunked documents can split important information across pieces, and the extra search step adds some latency. Knowledge bases also need ongoing maintenance to stay current. Finally, RAG reduces hallucinations but does not eliminate them: a model can still misread or misquote a retrieved source, so important answers should be checked against the citations.

If you are building a RAG system, frameworks such as LangChain and LlamaIndex handle much of the plumbing, from splitting documents to connecting a vector database and a language model. Start with a small, well-organized set of documents, test real questions, and improve chunking and retrieval before adding complexity. Clear instructions telling the model to answer only from the retrieved context, a core prompt engineering technique, make a big difference.

If you just want the benefits of RAG, you can use it today in AI chat apps. In Chat Smith, a free AI chatbot for Android, iOS, and the web, Web Search grounds answers in current information and Deep Research runs multiple searches to compile a sourced report, with models such as GPT-6 Sol, Claude Sonnet 5, and Gemini 3.5 Flash. Free users get limited use, and Chat Smith Pro raises those limits for $9.99 per month or $39.99 per year.

RAG is the right choice whenever answers need to reflect information the model was not trained on: recent events, private documents, or knowledge that changes often. It is also the best option when users need to see where an answer came from. For changing how a model writes or behaves, fine-tuning is usually a better fit, and for a one-time look at a few long files, simply sharing them in the conversation may be enough. For most real-world assistants, though, RAG has become the default way to make AI answers current, specific, and trustworthy.