Last week a senior technologist at one of the world’s largest payment companies told us that token costs were his number one priority. Even with fraud, agents, and the regulator to think about, what he wanted to discuss with us was the $2,000 per month (“conservatively”) that devs were spending on frontier lab tokens.
If you're responsible for AI infrastructure in a financial services firm, you need to know how much more of that spending to plan for. Which tasks justify a frontier model, which could run on something cheaper, and how do you keep usage within budget as more people start using agents?
The expense ultimately affects the P&L, even when it isn't disclosed as a separate inference figure. On LSEG's half-year call in July, David Schwimmer said token costs were "fully factored into our midterm margin guidance." For the people choosing models and designing workflows, though, the aggregate spend is only a starting point.
To decide where to cut costs, you need to connect that spending to the work being done. The price per token is part of the calculation, but so are the number of calls needed to finish a task and the amount of human checking afterwards.
Agentic coding is the heart of the cost problem
In the Bank of England and FCA's survey of AI in UK financial services, 70% of respondents said they monitored financial impacts such as cost and operational efficiency. That tells us firms are paying attention to the economics, but it doesn't establish whether they can attribute spend to particular models or workflows.
We suggest starting with a service you can define: resolving a customer query, supporting a fraud investigation, or completing a coding task. Agree what an acceptable result looks like, then measure the cost of producing it. You can compare models against that same task instead of assuming that a lower token price will make the whole service cheaper.
But what single workflow or activity should you start with? At the Evident AI Symposium in New York, JPMorgan's global CIO Lori Beer said coding assistants account for the majority of the bank's token spend. If that's true of the largest technology budget in banking, it's probably true of yours. Developers use frontier models to write, review and fix code, and agentic tools such as Claude Code can make dozens of model calls to finish one task.
Cheaper tokens can still mean a larger budget
We've covered this before in Cheaper tokens, bigger bills: you can pay less for each token while spending more overall because you use so many more of them. For a bank expanding from individual assistants to agentic workflows, like coding assistants, the extra model calls are part of the capacity and cost you need to plan for.
In McKinsey's enterprise FinOps survey in May, 93% of respondents had exceeded their AI budgets, and spending increased about fourfold when a use case expanded across the enterprise.
The researchers also found a 30-fold variation in token consumption for the same agentic task, depending on how it was built. You can spend very different amounts to get the same work done.
At a Computer Weekly roundtable in June, Deloitte's Hayley McKelvey described the commercial consequences: "Tokenomics is all we hear about right now. Organisations have signed consumption-based contracts with frontier AI labs, but many were not commercially ready for the financial impact."
With agents, it's easy to underestimate how much model use you're committing to.
A simple chatbot response may require a single model call. When you build an agent, you may call the model to plan a task, interpret a tool result, choose another action, and check the result. You pay for those intermediate steps as well as the answer the user sees.
Find the right model for each task
Controlling token costs doesn’t mean you have to step on the brakes. You can make a huge difference just by choosing the right model for each task.
Revolut's FinCrime team moved token-heavy reasoning steps to an open-weight model at roughly a third of the cost. In the case study published by its GPU provider, the team describes evaluating alternatives before choosing Kimi K2.5. One candidate lacked enough context capacity and another had quota problems under load, so price alone couldn't determine the choice.
You may not need reasoning on every step, either. Monzo's customer operations team found that "non-thinking models were as effective for triage as more expensive, slower alternatives". Testing that assumption let them avoid paying for a capability the task didn't require.
For a repeatable way to compare candidates, Santander's AI Lab uses a "Model Arena" to benchmark open-weight models against banking tasks. That work contributed to reducing its reported cost per response from €5 to three cents.
What changes when you're a bank
In a regulated firm, a cheaper model is still a model change, and your model risk function will want evidence before it goes live. The task-level testing above is that evidence. Build it once and use it twice: to choose the model and to validate it.
Routing also decides where your data goes. In our own pilot, described below, one of our principal architects put it simply: his Terraform state, cloud keys and platform token "never left the trust boundary they were issued in." For a bank, that boundary holds customer data and source code.
And if you control the routing, no single model provider becomes a dependency you can't exit, which your DORA or third-party risk team will ask about. The same goes for provenance: choose open-weight models against your own policy, and let the router enforce it.
Enforce choices with model routing
Once you've established which models can do the work, use routing to apply those choices consistently. You can reserve a frontier model for tasks that require it and use a cheaper or locally hosted model for others. In the Linux Foundation's State of Tokenomics survey, 86% of respondents were evaluating or using a model router (financial services accounted for 17% of its 472 respondents).
Routing also gives you somewhere to apply spending limits and collect usage records across the models you use. That helps when you need to compare the cost of a workload before and after changing its route.
We tested that approach with our own coding workloads in a month-long pilot of PaletteAI Inference Launchpad. From August 15 to September 14, up to 85 engineers used 41 billion tokens, with 97.5% served locally on an eight-GPU server. Engineers with access throughout the reporting period averaged about 46% less in direct Claude spending than colleagues outside the pilot. That's a comparison between groups, rather than a measured before-and-after saving for the same engineers.
We already had Claude's per-user, per-day reporting. The pilot gave us a way to route work to a local model while retaining Claude for selected tasks, and we reduced the daily frontier-model allowance from $200 to $50 per engineer. For your own comparison, include the local hardware and operating costs, and use the cached-token rates you actually pay.
Put workload costs and spending limits into daily use
Once you've tested the alternatives, make the model choice and budget part of how the service operates. Someone needs to be able to see the usage, explain it, and change the routing or limits when necessary.
- Route model calls through a shared gateway and record who made each call, which model they used, and how many tokens they consumed. Connect those records to the application or workload so the platform and finance teams can compare spending without manually reconciling separate supplier dashboards.
- Assign responsibility for managing that spend. In the FinOps Foundation's 2026 survey, 98% of practitioners said they managed AI spend, up from 63% a year earlier, and identified AI cost management as their top skills gap. Agree who can change a budget or routing policy, and when they should involve the service owner.
- Review cost alongside the outcome. A finance director can assess spending per resolved chat or fraud case, while a development team might track it per merged pull request. Include the work needed to check or correct the result, so a reduction in token spend doesn't simply mean more work for your staff.
- Set limits with the people using the service and decide what happens when they reach them. Can they continue with an approved cheaper model, request a higher allowance, or wait for the limit to reset? Test that behavior before relying on the budget control.
As time goes on, you may still choose to spend more overall as you make AI available to more people or use it for more work — the difference is, you’ll be able to explain the increase in terms of usage and outcomes, with evidence for why you chose each model.
Use our AI inference TCO calculator to compare hosted usage with local infrastructure costs, using your own volume and pricing assumptions. Then test the model on the work your team needs it to do.