How to estimate LLM API cost (and where the estimate goes wrong)

Estimating LLM spend comes down to one multiplication per model:

cost = (input tokens × input price + output tokens × output price) × requests

Everything hard about the estimate is deciding those three inputs. The prices come from the vendor. The rest is you.

1. Input tokens are bigger than you think

The message the user typed is usually the smallest part of the input. The full input on each turn is the system prompt, tool definitions, the retrieved context, and the whole conversation history so far, re-sent every time. A "short" question in a long thread can be 30,000 tokens of input.

This is why the calculator defaults to 1,500 input tokens rather than 100: it models a realistic turn, not a first turn. If you have logs, use the mean input tokens across all requests, not the median of the messages you looked at.

2. Output tokens are the expensive half

Across the 30 models in our catalog, output costs a median of 5.0× what input costs per token, and 15 of 30 charge at least 5×. One model prices output at 8.0× its input rate.

That single ratio decides whether your bill is dominated by what you send or what the model writes back. Long-context workloads (RAG, document analysis) tilt toward input; chatbots with verbose answers tilt toward output.

Output ÷ input on published list prices, Oct 15, 2026. It says nothing about quality — a cheap-to-generate answer is not a good answer.

3. Requests include the ones users never see

Retries, evaluation runs, classification steps, tool-calling loops and background jobs all bill. A workflow that calls the model four times per user action costs four times what a naive estimate says.

Practical approach: count model calls, not user actions, and add your retry rate on top.

Where the estimate goes wrong

• Caching. If you resend the same system prompt and context, most vendors bill a fraction — a median of 10% of the standard input rate across the 27 models that publish a cached rate. Ignore it and you overestimate on input-heavy workloads. Details and per-model rates →
• Price tiers. 7 models in the catalog charge a higher rate once a single request crosses a context threshold. A 250K-token document is not billed at the same per-token rate as a 2K-token prompt.
• Time-limited pricing. 3 models carry promotional or introductory rates with an end date. Modelling next quarter's budget on a promo rate is a common surprise.
• Time-of-day pricing. 2 vendors here discount off-peak hours. If your workload is batch and movable, that is free money.

More on tiers, promos and off-peak pricing →

A worked example

Support assistant: 1,500 input tokens, 400 output tokens, 200 requests per day (6,000/month). Using Qwen3.8-Flash at $0.120/$0.400 per 1M, that is $0.0003 per request and about $2.04 per month. On GPT-6 Astra — the most expensive output price in the catalog at $50 per 1M — the same workload is about $210 per month.

That spread — roughly 103× — is the reason to do the arithmetic before choosing a default. Whether the cheaper model can do the job is a question only your own evaluation can answer.

Run it on your own numbers →

Frequently asked questions

Keep going