RAG Application Cost Guide: Embeddings, Retrieval and Generation

Pricing verified 2026-07-30

RAG costs are not just answer generation. A real RAG app has document ingestion, embedding refreshes, vector storage, retrieval, optional reranking, and final answer tokens. Provider pricing pages rarely combine those steps for you.

Cost components

Component	What drives cost	How to control it
Embeddings	Documents ingested and refresh frequency	Deduplicate, chunk carefully, refresh only changed content
Retrieval	Queries and context sent to the model	Limit retrieved chunks and improve ranking
Generation	Input context plus output answer	Summarize context, cap answer length, cache static instructions

RAG support search

Assumptions: 20,000 requests/month, 4,200 input tokens, 700 output tokens, 35% cacheable input, 10% retry overhead. Scenario tables calculate USD rows only to avoid unsourced currency conversion.

Model	Monthly estimate	Annualized	Input/output driver
OpenAI gpt-5	$233.12	$2797.41	Output-heavy
OpenAI gpt-5-mini	$46.62	$559.48	Output-heavy
OpenAI gpt-5-nano	$9.32	$111.90	Output-heavy
OpenAI gpt-5.5	$778.47	$9341.64	Input-heavy

Limitations

This guide does not invent vector database or embedding prices because those vary by provider and deployment. Users should add their own storage and embedding rates from official vendor pages.

How to use this page in a real budget

This page is designed for independent developers, creator teams and small SaaS operators who need a quick but defensible estimate before committing to a model, workflow or monthly subscription. Start with the example table, then replace the assumptions with numbers from your own product: monthly active users, average requests per user, prompt length, expected output length, retry rate and the share of repeated context that may qualify for prompt caching.

The important habit is to estimate the completed task, not only the raw API call. A support chatbot may need one answer. A coding assistant may run several tool calls before it finishes. A RAG application may attach thousands of retrieved tokens before every response. A creator workflow may regenerate audio, music, images or video many times before one usable output is approved. When those loops are ignored, the advertised price per unit can look cheaper than the real production cost.

Common mistakes

Comparing only input-token prices while the workflow is output-heavy.
Ignoring retries, moderation repairs, JSON-format retries and agent tool loops.
Assuming cached input is always available even when prompts, retrieved documents or user-specific context change on every call.
Mixing USD and CNY rows without documenting an exchange-rate source.
Using public list prices for enterprise contracts, committed-use plans or regional marketplaces.

Practical cost-reduction tips

Reduce repeated prompt text first. Long system prompts, hidden instructions and pasted policies can become a silent monthly cost when multiplied by thousands of calls. Move static material into cacheable context when the provider supports it. Use smaller models for classification, routing, extraction and draft generation, then reserve stronger models for final decisions or customer-visible answers. Measure cost per successful task instead of cost per request, because a cheap model that causes many retries can become expensive.

For RAG Application Cost Guide: Embeddings, Retrieval and Generation, the best estimate usually comes from a three-scenario forecast: conservative, expected and high-growth. The conservative scenario protects cash flow. The expected scenario helps pricing decisions. The high-growth scenario reveals whether the architecture will stay profitable if traffic arrives suddenly from search, social media or a product launch.

FAQ

Are the prices guaranteed?

No. Tokencost records source-backed public pricing and community-dataset checks, but providers can change prices, discounts and regional availability. Verify the linked source before signing a client quote or procurement plan.

Why are some prices manual?

Some platforms use credits, regional plans, private enterprise pricing or bundled subscriptions. When a public unit price is not clear, Tokencost prefers a manual input instead of inventing a fake number.

Should I choose the cheapest model?

Not automatically. Compare quality, latency, retry rate, tool support and context length. The cheapest model for one workload may be the wrong choice for a workflow that needs reliable reasoning or long-context accuracy.

How often is pricing refreshed?

The pricing dataset is designed to refresh daily from source-backed data and to keep a visible last-verified date. Kimi rows are currently stored as CNY because the official Kimi platform publishes those prices in yuan.