When you build a chat feature on an AI API, it is natural to estimate cost per message. That estimate will be too low, because AI model APIs don't remember earlier messages on their own. To keep the conversation going, your app sends the whole history again with every new message.
How input grows
Imagine a chat with a 1,000-token system prompt, where each user message is about 200 tokens and each reply about 300. On the first turn, you send 1,200 tokens. On the second, you send the system prompt, the first exchange and the new message: 1,700 tokens. Each turn adds another 500.
By the tenth message, a single request carries 5,700 input tokens, even though the new message is only 200 of them.
If nothing were resent
10 × 1,200equals12,000 tokens
Earlier turns resent
500 × (0 + 1 + 2 + … + 9)equals22,500 tokens
Actual input sent
12,000 + 22,500equals34,500 tokens
What it costs
At $2 per million input tokens and $10 per million output tokens, the 34,500 input tokens cost $0.069 and the 3,000 output tokens cost $0.03, for about $0.10 per conversation. If history were not resent it would be about $0.054. Across 1,000 conversations a month, that is roughly $99 instead of $54.
Because the resent history grows with every turn, doubling the length of conversations more than doubles their cost.
What a chatbot costs across models
Model choice is the biggest single factor in the bill. Here is one realistic workload priced across several current models: 1,000 active users, 10 conversations each per month, 6 messages per conversation, an 800-token system prompt, 60-token user messages and 300-token replies, with full chat history. That is 10,560 input and 1,800 output tokens per conversation.
Input tokens
6 × 800 + 6 × 60 + 360 × 6 × 5 ÷ 2equals10,560
Output tokens
6 × 300equals1,800
Per conversation
10,560 ÷ 1M × $2.00 + 1,800 ÷ 1M × $10.00equals$0.0391
Per month
$0.0391 × 10,000 conversationsequals$391.20
- GPT-6 Luna: $19.56 a month
- GPT-5.6 Luna (previous generation): $42.72
- Gemini 3.5 Flash-Lite: $76.68
- Claude Haiku 4.5: $195.60
- GPT-6 Sol and Claude Sonnet 5: $391.20 each
- GPT-5.6 Terra and Gemini 3.1 Pro: $427.20 each
- Claude Opus 5.5: $782.40
The cheapest and most expensive options are about forty times apart for the same traffic. Mid-tier models cluster around $2 per million input tokens, so the difference between them comes mostly from output prices. Price is only half the decision: test each model on your own conversations, then pick the cheapest one that meets your quality bar. These prices were last checked on 29 September 2026, after OpenAI's GPT-6 Sol and Luna launch, and they change often.
Ways to keep it under control
- Prompt caching: many providers charge less for input that repeats from one request to the next, such as a long system prompt.
- Summarising: replace older turns with a short summary once a conversation gets long.
- Trimming: drop the oldest messages beyond a certain length, if the task allows it.
- Shorter replies: output tokens are priced higher and also become input on the next turn.
Longer conversations cost disproportionately more
Using the same workload on Claude Sonnet 5, but changing only the length of each conversation:
- 6 messages per conversation: $391.20 a month.
- 12 messages: $1,041.60 a month, 2.7 times as much for twice the messages.
- 20 messages: $2,312 a month, 5.9 times as much for about three times the messages.
Input per conversation grows from 10,560 tokens at 6 messages to 85,600 at 20, because every message resends everything before it. If your product encourages long sessions, such as tutoring, coaching or support, this effect will dominate your costs.
Scaling up the user base
Model choice matters even more at scale. For 10,000 users on the original 6-message workload, GPT-6 Luna costs about $195.60 a month. Before choosing a premium model for a high-volume product, test whether a cheaper model handles your conversations well enough, and consider routing only difficult conversations to a stronger one.
Estimating your own numbers
- Log a sample of real conversations for a week: the number of messages, and the input and output tokens the API reports for each call.
- Take the median conversation length, not the average, which a few very long sessions can distort, and separately note the longest 10%.
- Price both in the AI Feature Cost Calculator: the median gives your typical bill, the long tail tells you whether you need a length cap or summarisation.
- Repeat after launch, because real users behave differently from test users.
Tip: When you estimate costs, model a realistic conversation length rather than a single message. Averages from real usage logs are best once you have them.
Questions people ask
- Does prompt caching help with chat history costs?
- Yes. The system prompt and earlier messages are resent unchanged, so caching them can cut their input cost to around a tenth on models that support it. Output tokens and new messages are still charged in full.
- How much does it cost to run an AI chatbot?
- For 10,000 six-message conversations a month, from about $43 on the cheapest model tested here to about $427 on the most expensive, at September 2026 prices.
- Why does a longer conversation cost more than proportionally?
- The API doesn't remember earlier turns, so your app resends the whole history with every message. Input tokens grow with each turn.
- How can I reduce chatbot API costs?
- Use prompt caching for repeated input, summarize or trim older turns, cap reply length, and use the smallest model that passes your quality tests.
- How much more does a 20-message chat cost than a 6-message one?
- In our example on Claude Sonnet 5, about 5.9 times as much: $2,312 against $391.20 a month for the same 10,000 conversations.