- Default to RAG when your feature needs your customers' data, current facts or proprietary content. Fine-tune only when the model has the facts but behaves wrongly.
- Fine-tuning is no longer open to everyone. OpenAI has closed it to new users, and fine-tuned models stop working when their base model is retired.
- RAG's costs are mostly engineering and infrastructure, not embeddings. Fine-tuning's costs are training runs, higher inference rates in some cases, and repeating the work every time the model or data changes.
RAG (retrieval-augmented generation) means your app looks up relevant information at the moment of a request and hands it to the model along with the question. Fine-tuning means training a model further on your own examples so it changes how it behaves. For most SaaS AI features, RAG is the better first choice, because they usually need answers grounded in changing, customer-specific data. Fine-tuning suits a narrower problem: consistent format, tone or reasoning.
Below: when each one fits, what each costs to build and run, and the order we would try them in.
The short version: what problem are you solving?
The simplest test is to ask what is going wrong with the model's answers.
According to a third-party summary of OpenAI's guide, you should optimise context when the model lacks knowledge, has out-of-date knowledge, or needs proprietary information. You should fine-tune when the model has the facts but behaves wrongly, for example inconsistent formatting, wrong tone, or reasoning it doesn't follow.
Map that to a SaaS product:
- "Answer questions about this customer's contracts, tickets or inventory" is a knowledge problem. Use RAG.
- "Always return JSON in our exact schema and write in our brand voice" is a behaviour problem. Try prompting first, then fine-tuning.
- "Both" is common. That is fine, but build them in order, not at once.
Freshness favours RAG
SaaS data changes constantly. With RAG, you update the index when a record changes and later answers can draw on the new data.
A Manning textbook excerpt puts it plainly: use RAG when answers depend on changing facts, such as current inventory. Fine-tuned knowledge stays static until the model is retrained.
Fine-tuning has a platform risk you should price in
This is the part many founders have not heard yet. OpenAI is winding down its fine-tuning platform. New users can't access it, and eligible existing customers can create training jobs until January 6, 2027, unless the base model shuts down earlier. For many founders, the fine-tuning option on OpenAI has already closed.
The deprecation schedule is specific. From May 7, 2026, OpenAI blocked fine-tuning for organisations that had never run it. From July 2, 2026, it blocked organisations with no fine-tuned inference in the past 60 days. Inference on fine-tuned models is disabled only when the underlying base model is deprecated.
Base models retire too. Per the same OpenAI guide, gpt-4.1-nano-2025-04-14 and its fine-tuned versions shut down on October 23, 2026. gpt-4.1, gpt-4.1-mini and their fine-tuned versions shut down on January 14, 2027.
The practical meaning: a fine-tune carries both platform and base-model lifecycle risk. When the base model retires, so does the fine-tune built on it. Other providers, such as Google's Vertex AI, still list tuning, but the same lifecycle question applies to any fine-tune you build.
What RAG costs to build and run
RAG has three cost areas: embeddings, a vector store, and the engineering to make retrieval good.
Embeddings are cheap
OpenAI lists text-embedding-3-large at $0.13 per 1M tokens and text-embedding-3-small at $0.02. Embeddings are a small line item. The money in RAG goes on storage, queries and engineering.
The vector database sets a floor
Taking Pinecone as one example, its monthly minimum usage is Starter $0, Builder $20 flat, Standard $50 and Enterprise $500. A small production RAG feature therefore has a fixed floor of roughly $50 a month on Standard.
Read costs scale with data size. Per the same page, a query uses 1 read unit for every 1 GB of namespace size, with a minimum of 0.25 read units per query. Third-party pricing summaries list Standard at $0.33/GB per month for storage and $16 to $18 per million read units.
The takeaway: because read units follow namespace size, a larger namespace makes every query read more. How you split your data is a cost decision, not just a technical one.
Engineering is the real bill
OpenAI's accuracy guide notes that RAG needs retrieval tuned as well as LLM behaviour. That means you are tuning two things, not one, and budgeting engineering time for both.
What fine-tuning costs to build and run
Fine-tuning costs split into training, inference and upkeep.
Training
One tracker quoting OpenAI's docs lists training at $25.00 per 1M training tokens for gpt-4.1, $5.00 for gpt-4.1-mini and $1.50 for gpt-4.1-nano. Google lists Gemini supervised fine-tuning at $0.003 per 1,000 for Gemini 3.5 Flash Lite and $0.01 per 1,000 for Gemini 3.5 Flash. The unit there is "1,000 count", which a tracker reads as training tokens, equal to dataset tokens times epochs.
Inference
Fine-tuned models have their own inference rates. The same tracker's table shows fine-tuned gpt-4.1 inference at $3.00 input and $12.00 output per 1M tokens. On Vertex AI, a secondary review says tuned Gemini 3.x endpoints predict at 1.5x the base model rate for the life of the endpoint. That comes from a secondary source, so check Google's current docs before you rely on it.
Upkeep
Per OpenAI's guide, fine-tuning means rerunning the process and managing training and validation sets for every extra round. Every time your data or base model changes, you may repeat the cycle.
Side-by-side comparison
| Factor | RAG | Fine-tuning |
|---|---|---|
| Best for | Proprietary, changing or out-of-date knowledge | Consistent format, tone or reasoning |
| Data freshness | Update the index and answers can change | Static until retrained |
| Upfront cost | Engineering to tune retrieval as well as the model | Training and validation sets, plus training runs |
| Running cost | Vector database floor plus per-query reads | Retraining each round; inference rates may be higher |
| Platform risk | No fine-tuned model to retire | Tied to a base model's lifecycle and the provider's policy |
More retrieval is not always better
RAG is not a free upgrade on top of a tuned model. In OpenAI's own worked example, RAG added noise and confounded a model that had already learned the task through fine-tuning. Test each addition against a measurable baseline.
A practical ladder: evals, prompt, RAG, then fine-tune
OpenAI describes model optimisation as a mix of evals, prompt engineering and fine-tuning, which together form a feedback flywheel that produces better prompts and better training data. That supports a simple order of work:
- Write evals first. Collect real questions from your users, with the answers you would accept. Without this you cannot tell whether anything helped.
- Prompt well. Give clear instructions and a defined output format, and measure the result against your evals.
- Add RAG if failures come from missing, stale or private knowledge.
- Consider fine-tuning only if the model has the knowledge but still fails on format, tone or reasoning, and only after checking which provider and base model you can rely on.
If you do reach step four, the evals from step one feed the training and validation data.
What this means for a SaaS roadmap
If you are scoping an AI feature, plan for RAG as the core of the first release. Budget for retrieval quality, not just model calls. Keep the model behind an interface so you can switch providers when one retires a model, as OpenAI's schedule shows.
If you want help scoping this, see our AI development work and our project planner. For an example of our work, see BlueBox. You can also check our pricing.
FAQ
Is RAG cheaper than fine-tuning?
It depends on the workload. Embeddings are cheap, at $0.02 to $0.13 per 1M tokens on OpenAI, but a vector database adds a fixed floor of roughly $50 a month on Pinecone's Standard plan. Fine-tuning adds training runs, repeated retraining and, on Vertex AI, possibly higher inference rates. Your real cost depends on data volume and query traffic.
Can I fine-tune on OpenAI today?
Only if you are an eligible existing customer. New users can't access fine-tuning, and eligible existing customers can create training jobs until January 6, 2027, unless the base model shuts down earlier. Check the current docs before planning around it.
Can I use RAG and fine-tuning together?
Yes, but test the combination against your evals. OpenAI's own example showed RAG adding noise to a model already tuned for the task.
Should I fine-tune a model on my company's data?
Not for facts that change. Fine-tuned knowledge stays static until the model is retrained. For current product data and documents, retrieval is the better fit.
What should I build first?
Build evals, then a good prompt, then RAG if knowledge is the gap. Fine-tune last, and only for a behaviour problem you can measure.
