What is RAG in simple terms?

Retrieval-augmented generation (RAG) connects an AI model to your own documents and data, so every answer is grounded in your business — not the internet's average. When someone asks a question, the system finds the relevant passages in your material first, then asks the model to answer using only those passages, with citations back to the source.

How is RAG different from fine-tuning?

Fine-tuning adjusts a model's weights to change how it writes and behaves. RAG changes what the model knows at the moment it answers, by supplying the relevant source material with the question. If your problem is that the AI does not know your facts, RAG is the answer. If your problem is that it does not sound or behave the way you need, fine-tuning is the relevant tool. They solve different problems and can be combined.

When should we not use RAG?

If your knowledge fits in a handful of pages, if the question is numerical prediction or optimisation rather than language, if the answers live in a structured database that a query can answer exactly, or if your documentation is out of date and contradictory, RAG is the wrong tool. In the last case it will faithfully retrieve the wrong answer.

Insights

What is RAG? A plain-English guide for business

Q: Does RAG keep our data private?

It can, and privacy is a deployment decision rather than a property of RAG itself. Systems can run against a commercial API with contractual controls, inside your own cloud tenancy, or entirely on local hardware using local LLMs so no data leaves your infrastructure at all. Retrieval also respects existing permissions, so a user only ever sees passages from documents they are already entitled to read.

Lab0 Engineering · Published 1 August 2026 · Updated 1 August 2026

Retrieval-augmented generation (RAG) connects an AI model to your own documents and data, so every answer is grounded in your business — not the internet's average.

That single sentence is most of what a business needs to understand. The rest of this guide explains what problem it solves, how the mechanism works, what it costs, how it handles private data, and — the part suppliers usually skip — when you should not use it at all.

What problem RAG solves

General-purpose AI models have two failure modes that matter commercially. The first is hallucination: asked a question it cannot answer from what it learned, a model will often produce a fluent, confident, wrong answer rather than admitting the gap. In a customer-facing or operational context that is worse than no answer, because it is plausible enough to act on.

The second is staleness. A model's knowledge is fixed at training time and it has never seen your business at all. It does not know your product catalogue, your service terms, your internal procedures, last quarter's policy change, or the specific contract you signed with one particular client. Asked about any of them, it either declines or invents.

RAG addresses both by changing where the answer comes from. Instead of asking the model to recall, you give it the relevant material at the moment of the question and ask it to answer from that. The model contributes language ability; your documents contribute the facts. When the material does not contain an answer, a well-built system says so rather than filling the gap.

How it works

The mechanism is the same regardless of which vendor's tools are used underneath. There are five steps.

Collect the documents. Whatever holds the knowledge: policy documents, product specifications, support tickets, contracts, manuals, intranet pages, email archives. This is the source of truth the system will answer from.
Split them into chunks. Whole documents are too coarse to retrieve usefully, so each is divided into passages small enough to be about one thing but large enough to make sense on their own. How this is done has more effect on quality than most people expect.
Turn the chunks into embeddings. Each passage is converted into a numerical representation of its meaning and stored in an index. Passages about similar things end up near each other, which is what makes retrieval by meaning rather than by keyword possible.
Retrieve at query time. When a question arrives, it is converted the same way, and the system pulls the passages closest in meaning. Access controls apply here: a user only ever retrieves from documents they are already permitted to read.
Generate with citations. The question and the retrieved passages go to the model together, with an instruction to answer from that material only. The answer comes back with references to the source passages, so anyone can check it. That verifiability is the point.

RAG vs fine-tuning vs just using ChatGPT

These are three different tools that get compared as though they were competing versions of one thing.

Approach	What it does	Best for	Cost / effort
General chatbot, as-is	Answers from general knowledge learned at training time	Drafting, summarising, general questions	Lowest — a subscription and no engineering
RAG	Supplies your documents at question time, answers with citations	Anything that depends on your own facts, and where answers must be checkable	Moderate — a build project, then ongoing content upkeep
Fine-tuning	Adjusts the model itself to change how it writes and behaves	A consistent style, format or specialised task behaviour	Higher — needs curated training data and repeating when things change

The short rule: if the AI does not know your facts, that is a RAG problem. If it does not sound or behave the way you need, that is a fine-tuning problem. Businesses reach for fine-tuning to fix knowledge gaps far more often than they should, and it is the more expensive way to get a worse result — every content change means retraining, whereas with RAG you update a document.

What it costs and how long it takes

Anyone quoting a fixed price before seeing your material is guessing. The variables that actually move the number are these:

The state of your documents. Clean, current, well-structured content is the single biggest accelerator. Scattered, contradictory or scanned material means a preparation phase before anything else starts.
How many systems it has to reach into. One document store is straightforward. A document store plus a CRM plus a ticketing system plus a legacy database is an integration project.
Where it runs. A managed API deployment is faster to stand up than a fully local one on your own hardware.
How exact it has to be. An internal tool where staff can sanity-check answers is a different standard of evidence from a customer-facing system, and the evaluation work scales accordingly.
Ongoing running. Model usage, hosting and the indexing of new content are real recurring costs, though usually modest relative to the build.

Rather than publish a price list that would be wrong for most readers, we scope it. The first conversation is free, and if the honest answer is that your documents are not ready or the use case does not warrant it, that is what we will tell you.

Does RAG keep our data private?

It can. Privacy is a deployment decision, not a property of RAG itself, and there are three broad options.

The first is a commercial model API with contractual controls on retention and training. This is the quickest to deploy and appropriate for a great deal of internal work, but your queries and retrieved passages do leave your infrastructure.

The second is deployment inside your own cloud tenancy, where the index and the application sit within your boundary and your existing governance applies.

The third is fully local. Local LLMs running on your own hardware mean no data leaves your premises at any point — no external API, no third-party processor. This costs more in infrastructure and is the right answer where regulation, contractual obligations or the sensitivity of the material make it necessary. We build all three, and the choice should be made on your actual risk position rather than on a default.

One point that gets missed: retrieval must respect the permissions your documents already carry. If a user cannot open a file today, the system must not surface passages from it tomorrow. That is a design requirement from the first day, not something to bolt on later.

When not to use RAG

RAG is the wrong tool more often than the market suggests. Do not use it if:

Your knowledge fits in a few pages. If the whole body of relevant material is short enough to hand to a model directly, retrieval infrastructure adds cost and failure modes for nothing.
The question is numerical, not linguistic. Demand forecasting, pricing optimisation, anomaly detection in telemetry — these are modelling problems. Retrieving text does not help.
The answer lives in a structured database. If "how many orders shipped late last month" is a query someone could write, write the query. An exact answer beats a retrieved approximation.
Your documentation is out of date or contradictory. This is the important one. RAG will faithfully retrieve and confidently present the wrong answer, and it will look authoritative doing it. Fix the content first; the system inherits the quality of what you give it.
Nobody is willing to own the content. A retrieval system needs someone responsible for keeping the source material current. Without that, accuracy degrades quietly over months.

A RAG system running in production today

We do not just describe this pattern; we run it. Our AI operations page covers a production system where retrieval, human oversight and audit trails work together on live workloads. There is also a live interactive demo that walks through the retrieval flow step by step, so you can see what happens between a question and a cited answer rather than taking the description on trust.

Where to start

If you think RAG fits, the useful first step is not a technology decision. It is looking honestly at the documents you would want it to answer from, and asking whether a new colleague could do the job correctly using only those. If yes, the system probably can too. If not, that is the work to do first.

Our RAG development service covers scoping, build and production support, including local-LLM deployments. If you are an Irish business, part of the cost may be offsettable — our guide to AI grants in Ireland sets out the schemes and which one suits which stage.

The first conversation is free. We will tell you whether RAG is the right tool for your problem, including when it is not.

Last updated: August 2026

← Back to Insights