RAG vs fine-tuning for business chatbots
Last updated:
Two ways to give a chatbot your knowledge
A general-purpose large language model knows a lot about the world, but nothing about your opening hours, your return policy or the product you launched last month. To build a chatbot that answers questions about your business, you need a way to bring your own information into the conversation.
There are two common approaches. Retrieval-augmented generation, usually shortened to RAG, keeps your content outside the model and hands the relevant parts to it at question time. Fine-tuning changes the model itself by training it further on examples. They solve different problems, and mixing them up is one of the most common reasons business chatbot projects disappoint.
What retrieval-augmented generation is
RAG works in two phases. First, your content is prepared: web pages, documents and FAQs are split into short passages, often called chunks. Each passage is turned into an embedding, a list of numbers that represents its meaning, and stored in a searchable index.
Second, when a visitor asks a question, the system searches that index for the passages most likely to contain the answer. Those passages are placed in the prompt together with the question and a set of instructions, and the language model writes a reply based on them. The model has not memorised your content. It reads the relevant parts each time, much like an employee who checks the handbook before answering.
Because the facts live in the index rather than in the model, updating an answer is a content change, not a training job. You edit the page, index it again, and the next answer reflects the change. And because the system knows which passages it used, it can show where an answer came from, or say that it found nothing relevant.
What fine-tuning is
Fine-tuning takes an existing model and continues training it on a set of examples, typically pairs of an input and the output you want. The training adjusts the model's internal weights, so the new behavior becomes part of the model itself.
Fine-tuning is good at teaching a model how to respond: a consistent tone, a fixed output format, a classification scheme, or a narrow task it repeats many times. It is much less reliable as a way to teach facts. A fine-tuned model does not store your price list as a document it can look up. It absorbs patterns from the examples, and when asked about a detail it saw only once or twice, it may blend it with something else or fill the gap with a plausible guess.
Fine-tuning also needs prepared training data: a set of good, consistent examples that show exactly the behavior you want, plus a new round of training and evaluation whenever something important changes. Removing a fact the model has learned is not as simple as deleting a page; you generally have to retrain from corrected data.
RAG vs fine-tuning at a glance
The table below compares the two approaches on the points that matter most for a chatbot that answers questions about your business.
| Retrieval-augmented generation (RAG) | Fine-tuning | |
|---|---|---|
| What it changes | What the model reads before it answers | The model's weights, and so its default behavior |
| Keeping facts current | Edit or re-index the source; the next answer uses it | Needs new training data and another training run |
| Effort to update | Low: a content change | Higher: data preparation, training and testing |
| Tracing answers to sources | Possible, because the retrieved passages are known | Not directly possible; knowledge is spread across the weights |
| Controlling hallucinations | Instructions can limit answers to retrieved content and allow “I don't know” | Harder; the model may answer confidently from patterns |
| Data you need | Your existing pages, documents and FAQs | Curated examples of inputs and outputs |
| Removing content | Delete the source from the index | Retrain without that data |
| Best for | Factual answers from content that changes | Tone, format and narrow, repeated tasks |
Keyword, semantic and hybrid retrieval
The quality of a RAG chatbot depends heavily on its search step. If the right passage is not retrieved, even the best model cannot give the right answer. There are two main ways to search.
Keyword search matches the words in the question against the words in your content. It is precise with exact terms: product codes, model numbers, names, error messages and abbreviations. It struggles when the visitor uses different words than your page does, for example asking about “sending something back” when the page is titled “Returns policy”.
Semantic search compares embeddings instead of words, so it finds passages with a similar meaning even when the wording differs. It handles paraphrased and conversational questions well, but it can miss exact identifiers, because a code such as a part number carries little meaning on its own.
Hybrid search runs both and combines the results. A question like “Does the X200 fit my 2019 bike?” benefits from keyword matching on “X200” and semantic matching on the rest. For business content, which mixes everyday language with names and codes, hybrid retrieval is usually the most robust choice.
When fine-tuning makes sense
Fine-tuning is not the wrong tool; it is a different one. The two approaches can also be combined: a model fine-tuned for tone or format can still use retrieval to get its facts.
Fine-tuning is worth considering when one or more of these apply:
- You need a very consistent output format, such as structured data for another system.
- The task is narrow and repeated at high volume, such as sorting incoming messages into fixed categories.
- You want a specific style or tone that instructions alone do not produce reliably.
- You have a large set of high-quality examples and the capacity to retrain and test when things change.
Preparing your content for a RAG chatbot
For most website chatbots, clear instructions do the job people hope fine-tuning will do, and retrieval supplies the knowledge. But retrieval can only find what is in your content, and it works best when that content is clear. A few habits help:
- Start from pages written for customers: FAQs, product and service pages, pricing, shipping, policies and help articles.
- Leave out pages that only add noise, such as archives, tag pages, login pages and outdated announcements.
- Remove or update old documents. If two sources contradict each other, the chatbot may retrieve the wrong one.
- Write Q&A pairs for questions where the exact wording matters, such as cancellation terms or warranty conditions.
- Keep to one topic per page or section where you can, so each passage makes sense on its own.
- Re-index after important changes, such as new prices or opening hours.
- Before you go live, test with the questions real visitors ask, including vague ones and ones your content does not answer.
How intoCHAT does it
intoCHAT is built on retrieval. When you create an agent, you train it on your own content: you paste your website URL and choose which pages to read, upload files such as PDFs, Word documents, spreadsheets or presentations, and add text snippets or Q&A pairs. intoCHAT splits that content into passages and creates embeddings for them.
When a visitor asks a question, intoCHAT runs hybrid retrieval, combining semantic and keyword search over your content, and passes the best passages to the AI model together with your instructions. Built-in rules keep answers grounded in your sources and resist prompt injection, and the agent says when it doesn't know instead of inventing an answer.
intoCHAT doesn't fine-tune models on your content. In intoCHAT, training an agent means indexing your content for retrieval, not changing the model. The AI model is an OpenAI model chosen and managed by intoCHAT, and passages of your content are sent to OpenAI to create embeddings and generate answers. This is also why updates are quick: your knowledge lives in sources you control, not inside a model.
- Q&A answers take priority over other sources, so you can fix the wording of important answers.
- Starring a source makes it rank higher in search, with no retraining needed.
- You can edit, preview, download, delete or retrain each source. Retraining is manual, so retrain a source after its content changes.
- The playground lets you test the agent with real questions before it goes live.
Choosing the right approach
If your chatbot's job is to answer customer questions from information you publish, start with retrieval. It uses the content you already have, stays current when you update that content, and keeps answers tied to your sources. Consider fine-tuning only when you need a specific behavior that instructions cannot produce, and you have the data and time to maintain it.
The quickest way to see retrieval in practice is to try it on your own website: the free preview tool builds a temporary agent from five of your pages, and you can create your intoCHAT agent for free when you want to go further.