How to Cut Your AI Chatbot's API Bill: Model Choice, Caching and Batching

Most business owners who launch an AI chatbot or AI agent worry about one thing a few months later: the monthly bill from the AI provider. The good news is that the bill is not fixed. The same chatbot, answering the same customers, can cost very different amounts depending on three decisions made during the build: which model handles each task, whether repeated instructions are cached, and whether non-urgent work is sent in batches. This guide explains all three in plain terms, using Anthropic's published Claude API pricing as the example.

Start with how the bill is calculated. AI providers charge per token, which is roughly a piece of a word. You pay for input tokens, meaning everything you send to the model, and output tokens, meaning everything the model writes back. Prices are quoted per million tokens. On Anthropic's current pricing page, Claude Opus 5.5 costs 4 US dollars per million input tokens and 20 dollars per million output tokens. Claude Sonnet 5.5 costs 2 dollars input and 10 dollars output. Claude Haiku 5.5, the smallest current model, costs 0.10 dollars input and 0.50 dollars output for prompts up to 100,000 tokens. That is a gap of 40 times between the largest and smallest of these models.

The first lever is model choice, and it is the biggest one. Many chatbots are built on a single large model for everything, because that is the easiest way to get a demo working. But most customer messages are simple. Asking for opening hours, checking an order status, or booking a slot does not need the most capable model. A sensible design sends routine questions to a small, fast model and passes only the hard cases, such as a complicated complaint or a multi-step quote, to a larger one. This is often called model routing. It needs testing, because a small model that gives wrong answers is not a saving, but for well-defined tasks the difference in cost is large.

The second lever is prompt caching. Every chatbot carries a block of standing instructions: who it is, how to talk, your policies, your price list, maybe a few pages of FAQs. Without caching, that block is sent and billed in full on every single message. With prompt caching, the provider stores that repeated block and charges a much lower rate when it is reused. According to Anthropic's pricing page, reading from the cache costs one tenth of the normal input price on most models, and only one twentieth on Opus 5.5 and Sonnet 5.5. Writing to the cache costs a little more than normal input, 1.25 times for a five minute cache or 2 times for a one hour cache, so caching pays off when the same instructions are reused often.

Here is a simple illustration of why that matters. Imagine a support bot whose instructions and FAQ add up to several thousand tokens, while a typical customer question is only a few dozen tokens. Without caching, almost all of your input spend goes on re-sending the same instructions. With caching, that large repeated block is billed at a fraction of the price after the first write, and the per-message cost falls sharply. The exact saving depends on your traffic pattern, so it should be measured on real usage rather than assumed.

The third lever is batching. Not every AI task needs an answer in two seconds. Summarising yesterday's call recordings, tagging last week's support tickets, writing product descriptions for a new catalogue, or scoring a list of leads can all wait. Anthropic's Message Batches API is built for this. Its documentation says batch requests cost 50 percent less on both input and output tokens, that most batches finish within an hour, and that results are available once all requests complete or after 24 hours, whichever comes first. One batch can hold up to 100,000 requests. The pricing page also confirms that the batch discount and prompt caching can be combined.

So how does this look in practice? A dental or physiotherapy clinic in Delhi or Dubai might run a website and WhatsApp assistant that answers questions about timings, fees and preparation for procedures. Those answers can come from a small model with the clinic's policies cached, while a request to reschedule a surgery or a billing dispute is routed to a larger model or a human. Each night, an n8n workflow can send the day's conversations as a batch to produce a short summary and a list of patients who need a call back.

A real estate agency in Gurugram, London or Texas gets dozens of property enquiries a day from portals and its website. A chatbot can collect budget, location and timeline using a small model, with the agency's listing rules and FAQs cached. The longer, more careful work, such as writing a personalised follow-up based on several messages, can go to a more capable model. Weekly lead scoring across the full enquiry list is a natural fit for batch processing, because nobody needs that report within seconds.

An e-commerce brand on Shopify can use the same pattern. Order tracking and return policy questions go to a small model. Product description writing for a new collection of hundreds of items goes through the batch API at half price. Customer reviews can be summarised in bulk overnight to show which products are drawing complaints.

There are limits and risks to be honest about. Cheaper models are less capable, so routing must be tested on real conversations, and there should always be a clear path to a human. Prices change, and Anthropic's own pricing page notes that newer Claude models use a tokenizer that produces roughly 30 percent more tokens for the same text than older models, so a lower headline price does not translate one to one into a lower bill. Haiku 5.5 also has a higher price tier for prompts over 100,000 tokens, which matters if you feed it very long documents. Caching only helps when the repeated part of the prompt stays identical and is reused before the cache expires. And batch processing is unsuitable for anything a customer is waiting on. Finally, the AI model is only one part of running cost. Phone lines, WhatsApp messaging fees, hosting and monitoring add their own charges.

This is where Legacy Services can help. When we build an AI chatbot, AI calling agent or custom AI agent, we design for running cost from the first week, not after the first bill. In practice that means mapping your real customer questions, deciding which ones a small model can safely handle, setting up routing so harder cases reach a more capable model or a member of your team, and structuring your instructions and knowledge base so they can be cached. We then log token usage per conversation so you can see what each chat actually costs.

For background work, we use n8n workflows to move non-urgent jobs into scheduled batches: nightly call summaries, ticket tagging, lead scoring, catalogue content and report generation. If you already run a chatbot and feel the bill is higher than it should be, we can review your current setup, measure where the tokens are going and suggest specific changes before you rebuild anything.

If you want to understand what your AI chatbot or agent should cost to run, and where the savings are in your own setup, book a free 30-minute strategy call with Legacy Services at legacyservices.in/contact. Bring your current usage or your plans, and we will walk through the options with you in plain language.