How to plan the ongoing cost of an AI system

An AI project needs a budget for both development and ongoing operation. The operating cost depends on how often the system is used, how much information it processes, where it runs, and what support it needs. Planning around expected usage helps a team understand those costs before rollout.

This guide covers what goes into the operating budget, how to estimate it from expected usage, and how to check the estimate once the system is live. The worked example follows a made-up insurance broker that uses AI to sort and summarize incoming claim documents.

What you pay for after launch

Most operating costs fall into five groups.

• Model usage. Hosted model APIs usually charge for the amount of text sent in and generated, measured in tokens. Images, audio, and long documents count too. If you run an open model on your own infrastructure, you pay for the servers that run it.
• Hosting and infrastructure. Servers, databases, document storage, search indexes, queues, and the records the system keeps.
• Integrations. Connections to systems like a CRM, ERP, or document store. Some carry their own licence or API fees, and all of them need upkeep.
• People. Time spent reviewing outputs, handling exceptions, answering user questions, and keeping the system up to date.
• Testing and monitoring. Re-running evaluations after changes, watching quality and spend, and looking into problems when they come up.

Model usage gets the most attention, but it's only one line. For many workflows, review time and maintenance matter just as much.

Start from expected usage

A useful estimate starts with how the system will be used. Four questions cover most of it:

• How many tasks? The number of documents, emails, requests, or conversations per month, and how that changes in busy periods.
• How big is each task? The length of the inputs, the number of pages, and how much reference material goes along with them.
• How many model calls per task? A workflow often has several steps. Each one counts.
• How often does a step repeat? Retries after errors, follow-up questions, and re-runs when a reviewer asks for changes.

Here's how that works at the broker. Suppose the team receives about 1,500 claims a month, with a busier stretch after winter storms. Each claim arrives with several documents: a claim form, photos, an adjuster's notes, and sometimes a repair estimate. The workflow makes four model calls per claim:

1. Classify each document.
2. Extract key fields such as dates, policy number, and claimed amount.
3. Check the claim against the relevant policy terms.
4. Draft a summary for the claims handler.

That's around 6,000 calls in a normal month. The calls differ a lot in size. Classification sends a short excerpt and gets back a label. The policy check might send the full policy wording every time, which can make it the largest line in the estimate.

Build the estimate one step at a time in a spreadsheet. For each step, record the typical input size, the typical output size, the number of calls per task, and the model it uses. Multiply by the provider's current published rates, then by monthly volume. Add a second set of figures for a busy month. Rates change, so recheck them before each budget cycle.

Where the size of each request comes from

The amount of text in each call often matters more than the number of calls. Several things add to it:

• Instructions. Long instructions sent with every call add up across thousands of calls.
• Reference material. Retrieval that returns ten documents when two would do.
• History. Conversations that carry every previous message forward.
• Output length. Responses that are longer than the next step needs.

There are practical ways to reduce each one. Send only the relevant section of a long document. Use the provider's caching features for content that repeats across calls, where they're available. Use a smaller model for simple steps like classification. Ask for structured output with only the fields you need. Run work that isn't urgent in batches if the provider offers lower rates for it.

At the broker, the policy check is the obvious place to start. If the system pulls only the sections of the policy that relate to the type of claim, that step gets much smaller.

Each change can affect the quality of the result, so test it on real examples before you adopt it. We explain how in how to evaluate cost and quality in an AI workflow.

Hosting choices change the shape of the bill

Where the model runs changes how costs behave.

With hosted model APIs, cost rises and falls with usage. That keeps spending low while volume is small, and there's little infrastructure to manage. The trade-off is less control over model versions, and your data is processed under the provider's terms.

With open models on your own infrastructure, most of the cost is fixed capacity: the servers or GPUs, plus the people who keep them running. That can suit steady, high volume or strict rules about where data can go. Capacity costs the same whether the system is busy or idle, so uneven demand makes it harder to justify.

Many teams use a mix, such as an open model for high-volume classification and a hosted model for the harder steps.

Questions that help decide:

• Does the data have to stay in a particular environment or region?
• Is the volume steady, or does it spike?
• Who will patch, monitor, and scale the infrastructure?
• How much does it matter to control when the model changes?

If running your own models is on the table, our model selection and deployment work covers that comparison.

Integrations and maintenance keep costing after launch

A system that works on launch day still needs looking after. Plan for:

• Changes in connected systems. A new CRM field, an ERP upgrade, or a changed API can break an integration.
• Model updates. Providers update and retire models. Each change means re-testing, and sometimes adjusting instructions.
• Business changes. New products, new policy terms, and new document formats all need updates to rules, instructions, or reference material.
• Test set upkeep. New kinds of inputs appear over time and should be added to the examples you test against.
• Review time. People checking outputs is a real cost. It often falls as the system proves reliable on a given task, but budget for it at the start.

At the broker, a change to the claim form layout would affect extraction. A new policy product would need new reference material for the policy check. Neither is unusual, so both belong in the plan.

Give the system a named owner and a maintenance allowance in the budget. Much of the ongoing effort sits in the connections to existing systems, which is why we treat systems integration as part of the project from the start.

Check the estimate against real use

A pilot is the right time to replace assumptions with measurements. Track:

• Actual calls per task and the size of each one.
• How often steps retry or fail.
• Review and correction time per task.
• Cost per completed task, alongside total spend.

Cost per task is the number to watch. Total spend will rise as usage grows, which is expected. A rising cost per task tells you something changed: longer inputs, more retries, or a step that's pulling in more reference material than it needs.

Before rollout, set a few controls:

• A monthly budget with an expected figure and a busy-month figure.
• Usage limits and spending alerts, with the provider or in your own system.
• A monthly review for the first few months, then quarterly.
• An agreed threshold that triggers a redesign, such as cost per task rising past a set figure.

With those in place, the budget becomes something the team can check and adjust. The numbers come from real use, and changes show up early enough to act on.