In every workshop I run, the same question comes up: should we build a retrieval system or fine-tune the model on our data? The sharper question is: what exactly are you trying to fix?
If the problem is knowledge, use retrieval
The model does not know your internal regulations, your product prices or your leave policy. That is knowledge, and knowledge is supplied at question time rather than baked into weights.
Retrieval's decisive advantage is that updating information means updating a document — not retraining. If your policy changes tomorrow, replacing the file is enough.
If the problem is style, fine-tune
You want the model to always write in your brand voice, or always emit a specific structured format, or handle the vocabulary of a very specialised domain. That is behaviour and style, not information.
But note: fine-tuning is costly and needs good data, and many we-need-fine-tuning cases are solved by a well-written prompt and a handful of examples.
The practical answer in most projects
Start with a good prompt. If that is not enough, add retrieval. If that still is not enough, consider fine-tuning. This order saves most teams months and budgets.
In mature projects all three come together: a model tuned for style, retrieving knowledge at question time, under a carefully designed prompt.
