RAG vs fine-tuning for business apps

When a business wants an AI model to “know” its information — policies, products, documents, records — there are two main approaches: retrieval-augmented generation (RAG) and fine-tuning. They’re often talked about as alternatives, but they solve different problems. Picking the wrong one is one of the most common and expensive mistakes we see.

Short version: use RAG to give a model knowledge. Use fine-tuning to change its behaviour. Most business apps need the first; a few need both.

What RAG does

RAG leaves the model unchanged. Instead, when a question comes in:

Your system searches your own content for the passages most relevant to the question.
Those passages are added to the prompt.
The model writes its answer using them.

Think of it as an open-book exam. The model doesn’t need to memorise your returns policy; it reads the relevant page each time it answers.

Where RAG shines

Knowledge that changes. Update a document and the next answer uses the new version. No retraining.
Answers that must be traceable. Because you know which passages were used, the app can show sources.
Lots of material. Thousands of documents are fine; you only send the relevant pieces.
Access control. You can restrict retrieval to what each user is allowed to see.

What RAG needs to work well

RAG quality depends mostly on retrieval quality. That means clean content, sensible chunking (how documents are split up), good search — usually a mix of semantic vector search (for example with pgvector) and keyword search — and evaluation to check the right passages are being found.

What fine-tuning does

Fine-tuning continues training a model on your own examples, adjusting its internal weights. It changes how the model responds — tone, format, the kind of reasoning it applies to a narrow task.

Think of it as training a new employee on examples of good work, rather than handing them a manual.

Where fine-tuning shines

Consistent format or style. Always producing a particular report structure or brand voice.
Narrow, repeated tasks. Classifying tickets into your categories, extracting fields from a specific kind of document.
Cost and speed at scale. A smaller fine-tuned model can sometimes do a narrow task as well as a larger general model, more cheaply.

Where it struggles

Facts that change. New information means retraining.
Traceability. Knowledge absorbed during training can’t be cited.
Data requirements. You need a good set of high-quality examples, and preparing them is real work.

Side by side

	RAG	Fine-tuning
Best for	Giving the model knowledge	Changing the model’s behaviour
Updating information	Edit the documents	Retrain the model
Can cite sources	Yes	No
Upfront effort	Content preparation and retrieval	Collecting and labelling training examples
Typical first choice for business Q&A	Yes	Rarely

How to decide

Ask these questions in order:

Is the problem that the model doesn’t know our information? Start with RAG.
Is the problem that the model knows enough but responds in the wrong way? Try better prompting and examples in the prompt first. They’re cheaper and faster to change.
Is the behaviour still inconsistent at scale, or is cost/latency a problem for a narrow task? Now consider fine-tuning.
Do you need both specific knowledge and very specific behaviour? Combine them: a fine-tuned model that also uses retrieval.

In our experience, the large majority of business assistants — customer support, internal knowledge bases, school and campus assistants — are best served by RAG plus good prompting and evaluation.

What we use in practice

For client assistants and our own products we default to RAG: content preparation, hybrid search over pgvector, and evaluation sets of real questions that every change is tested against. We reach for fine-tuning when a narrow task is well defined, high volume, and the examples to train on already exist.

If you’re deciding for your own product, tell us about it — the scoping call is free. Or read about our AI development service.