What LLM integration really costs in 2026: cloud, on-prem, and everything nobody quotes you for

Every LLM project starts with a demo that works. Someone pastes a contract into a chat window, asks for the risky clauses, and gets a genuinely good answer in nine seconds. The room goes quiet. Then somebody asks the only question that matters: what would it cost to do that ten thousand times a month, on our documents, without it ever being confidently wrong in front of a customer?

That question has three answers, and most quotes only cover the first one. There's the cost of the model — the bit everyone benchmarks. There's the cost of the infrastructure it runs on, which is where the cloud-versus-on-prem argument lives. And there's the cost of everything around the model: preparing your data, checking its work, wiring it into the systems you already use, and keeping it good after launch. That third bucket is usually the largest, and it's the one that surprises people in month four.

This is the honest version. Real numbers, checked in July 2026, with the arithmetic shown so you can redo it with your own volumes. Prices move fast in this market — treat every figure here as a snapshot, not a quote.

First, what an LLM actually is — and what you're paying for

A large language model is a system that predicts text. You give it some words, it produces the words that should come next. That sounds trivially small for something that can read a contract, but the prediction is so well-informed that it behaves like reasoning for a huge range of tasks — summarising, classifying, extracting, drafting, answering questions about documents it's just been shown.

Two things follow from that, and they drive every number in this article.

The first is that an LLM has no memory of your business unless you give it one. Out of the box it knows what it learned in training, not what's in your CRM. Everything useful you'll build involves feeding it your own information at the moment of the question — a technique called retrieval-augmented generation, or RAG. That's an engineering job, not a model job, and it's where much of the real work sits.

The second is the unit of cost: the token. A token is a chunk of text, roughly three-quarters of an English word — so a thousand tokens is about 750 words, and a typical page of a contract is around 500 tokens. You are billed for tokens in (everything you send: the instruction, the retrieved documents, the conversation so far) and tokens out (what the model writes back). Output tokens cost several times more than input tokens across every provider. Once you know roughly how many tokens a single transaction consumes, you can price the whole thing on a napkin — and by the end of this article you'll be able to.

Two roads: rent a model, or run one

There are exactly two ways to get a model into your business, and choosing between them is a business decision dressed up as a technical one.

Rent one. You call a model over the internet through an API — Claude, GPT, Gemini — and pay per token. No hardware, no capacity planning, and you get whatever the newest model is the week it ships. Your data leaves your perimeter to be processed, under whatever contractual terms you've agreed with the provider.

Run one. You take an open-weights model — Llama, Qwen, Mistral, Gemma, DeepSeek — and run it on hardware you rent by the hour or own outright. Nothing leaves your building. You control the version, so nothing changes underneath you without your say-so. In exchange you now own a capacity problem, a hardware problem, and a staffing problem.

There's a third road that most serious deployments end up on: both. A routing layer in front of several models sends each request to the cheapest one that's good enough — a small model for routine classification, a frontier model for the genuinely hard calls, an on-prem open-weights model for anything containing personal data. That's more work to build, and it's usually where the biggest cost savings come from.

What the cloud road costs

Here's what the major providers charge to process a million tokens, as of late July 2026. A million input tokens is roughly 750,000 words — about nine copies of The Great Gatsby.

Model	Input, per 1M tokens	Output, per 1M tokens
Claude Haiku 4.5	$1.00	$5.00
Gemini 3.6 Flash	$1.50	$7.50
GPT-5.6 Terra	$2.50	$15.00
Claude Sonnet 5	$3.00	$15.00
Claude Opus 5	$5.00	$25.00
GPT-5.6 Sol	$5.00	$30.00

Sources: Anthropic, OpenAI, Google, as published in July 2026. Every provider also offers two large discounts that most teams forget to use: cached input, where text you send repeatedly — a long system instruction, a fixed set of reference documents — is charged at roughly a tenth of the normal rate, and batch processing, which takes 50% off everything if you can tolerate results arriving within hours rather than seconds.

Now the arithmetic that actually matters. Take a customer-support assistant handling 10,000 conversations a month. Assume four exchanges per conversation, and that each exchange sends about 6,000 tokens in (your instructions, the retrieved help-centre articles, the conversation so far) and gets 300 tokens back.

That's 24,000 input and 1,200 output tokens per conversation — 240 million input and 12 million output tokens a month. Run that through the table:

• Claude Haiku 4.5: (240 × $1) + (12 × $5) = $300/month
• Claude Sonnet 5: (240 × $3) + (12 × $15) = $900/month
• Claude Opus 5: (240 × $5) + (12 × $25) = $1,500/month

Two things should jump out. First, the spread between a cheap model and a frontier model is five-fold, which sounds enormous until you notice the absolute numbers are all small — this is not the line item that will hurt you. Second, most of that input is the same text every time: the instructions and the help articles barely change. Turn on prompt caching and the Sonnet figure drops to roughly $450/month, a saving of about half, for a configuration change rather than a rebuild.

What the model costs is rarely the problem. What surrounds it is. Add application hosting, a vector database for retrieval, logging and monitoring, and the retries you'll pay for when a call fails halfway — and the platform bill around a modest LLM feature routinely exceeds the model bill. Worse, it scales with usage, not with your headcount, which is exactly the failure mode OWASP flags as unbounded consumption in its LLM risk list. Put a hard spending cap and per-user rate limit in before you launch, not after somebody scripts a loop against your endpoint.

What the on-prem road costs

Running your own model splits into renting GPUs by the hour and buying them outright. Start with renting, because almost everyone should.

Here are on-demand hourly rates per GPU, taken directly from two providers' public pricing pages in July 2026:

GPU	RunPod (secure cloud)	Lambda
L40S 48GB	$0.99	—
A100 80GB	$1.49	$2.79
H100 PCIe 80GB	$2.89	—
H100 SXM 80GB	$2.99	$3.99–$4.29
B200	$5.89	$6.69–$6.99

The spread between providers for identical silicon is large — often two-fold — and it maps onto real differences in networking, support, and whether your workload lands on dedicated or shared infrastructure. Cheaper is not automatically worse, but for anything touching regulated data, read the contract before the price list.

Which card you need depends entirely on model size. A small open-weights model quantised down to run efficiently fits comfortably on a single L40S. A mid-sized model wants an A100 or H100. The largest open models need several H100s working together — and at that point you're renting a machine, not a card.

Now buying. As of 2026, an NVIDIA H100 80GB PCIe card costs roughly $25,000–$30,000, the higher-bandwidth SXM5 variant $35,000–$40,000, and a complete eight-GPU DGX system $350,000–$400,000+. Those are the numbers people quote. Here are the ones they don't.

Power. A full eight-GPU H100 system draws about 10.2 kW. Run it continuously for a year and that's roughly 89,000 kWh. UK business electricity in 2026 runs about 20–26p per kWh for an SME, so you're looking at £18,000–£23,000 a year in electricity alone — before cooling, which for dense GPU racks is not a rounding error.

Everything else. Racking and networking, cooling infrastructure, spare parts, and the fact that nobody sells you an engineer to keep it running at 2am. And depreciation: this hardware loses 15–20% of its value a year, faster when a new generation ships.

Put it together and the break-even is sobering. Buying a $25,000 GPU instead of renting one at $3/hour breaks even at about 8,333 hours — 347 days of genuine, round-the-clock use. Factor in power, cooling, racking, and depreciation and published analysis puts the real break-even at 18 months or more of near-continuous utilisation. Below roughly 40 hours of use per week, renting wins outright and it isn't close.

Almost no business workload runs a GPU flat out. Support tickets arrive during office hours. Document processing is bursty. The honest version is that most companies who buy GPUs for a business application end up with expensive metal idling most of the week.

So which one should you pick?

Cost is the wrong first question. Work through these instead, roughly in order:

• Regulation and data residency. If your legal or compliance position genuinely forbids sending data to a third party — and for some fintech, legal, health, and public-sector work it does — the decision is made for you, and the on-prem premium is the price of being allowed to do the thing at all. We've built exactly this for KYC compliance workflows and contract risk scoring. But check the actual requirement rather than assuming: many providers now offer contractual terms, regional processing, and zero-retention options that satisfy more auditors than people expect.
• Volume, and how predictable it is. Owning only makes sense with high, steady, continuous demand. Spiky or seasonal traffic is exactly what renting is for.
• Latency. If you need consistent sub-second responses inside a physical process, local inference can beat a round trip to a data centre. For a support assistant or a document pipeline, it makes no difference.
• Who will run it at 3am. This is the one people skip. Self-hosting means somebody owns uptime, capacity, model updates, and disaster recovery. If that person doesn't exist and you're not hiring them, you're not really choosing on-prem — you're choosing an outage.
• Model velocity. Frontier models improve every few months, and on the cloud road you get those improvements by changing a string in a config file. The GPU you bought in January is the GPU you have in December. That's a feature if you value stability and a real cost if you value capability.

The costs that never appear on the quote

Here's the part that decides whether the project works, and it has almost nothing to do with which model you picked.

Data validation and preparation. The single biggest predictor of whether an LLM project succeeds is the state of the data it has to work with — not the model. Documents in six formats with inconsistent naming, a knowledge base last updated in 2023, three systems that each hold a slightly different version of the customer record. None of that is an AI problem, and all of it lands in the AI project. Expect real weeks here, and be suspicious of anyone who scopes a build without asking to see your data first.

Annotation and labelling. Building anything you can measure requires examples with known-correct answers — and for many tasks, a human has to produce them. Managed annotation runs roughly $0.01–$0.50 per label for straightforward work, and vendor rates land around $6–12 per hour for general tasks and $50–100 per hour for specialist domains like medical or legal review. Those ranges are wide because complexity varies enormously; treat them as orientation, not a quote. The catch is that the labels you most need usually can't be outsourced at $8/hour, because only your senior underwriter knows what a correct answer looks like. Their time is the real cost, and it's the one nobody puts in the budget.

Fine-tuning versus prompting versus RAG. Fine-tuning — further training a model on your own examples — is the first thing people ask for and usually the last thing they need. Work through it in order. Can a better-written instruction solve it? Then do that; it's free and you can change it in a minute. Does it need to know your specific content? Then it's RAG, and that's an engineering build. Do you need a consistent output format at enormous scale, a specialist voice, or a small cheap model to match a big expensive one on one narrow task? Now fine-tuning earns its cost. It needs a labelled dataset, a training run, and an evaluation harness to prove it actually improved anything — and it must be redone whenever your requirements move.

Human in the loop. For any decision with real consequences, some proportion of the model's output should be reviewed by a person before it takes effect. This is not a temporary scaffold you remove after launch — it's a permanent line item, and it should appear in the business case from day one. The good news is that the proportion should fall over time: as your evaluation data shows which categories the model handles reliably, you route those straight through and reserve human attention for the ambiguous and the expensive. That's a measurable curve, and it's one of the things you should hold your delivery partner to.

Integration with everything else. The model is a small part of the system. The rest is plumbing: your CRM, Slack or Teams, an ERP, authentication, permissions so the assistant can't surface a document the asker isn't allowed to see, audit trails, rate limits, and graceful behaviour when a third-party API is down. On most builds we do, the LLM-specific work is a minority of the engineering. Anyone quoting you a price based purely on model complexity hasn't costed the integration.

Proving it works — and knowing when it stops

Pick the number the integration exists to move before you build it. Not "improve customer experience" — first-response time, cost per resolved ticket, contracts reviewed per analyst-day, percentage of invoices processed without a human touch. If you can't name it, you can't tell success from an expensive demo, and you'll end up arguing about vibes in six months. Our beginner's guide to KPIs covers how to choose one that survives contact with reality.

Then build an eval harness — a fixed set of real examples with known-good answers that you run the whole system against automatically. This is the single highest-value piece of engineering in an LLM project and the one most often skipped. Without it you cannot tell whether today's prompt change helped or quietly broke something for a category of users you don't hear from.

And run it continuously, not once. Three things drift underneath a working system. Your data changes — new products, new policies, new document formats. Your users change, finding phrasings your test set never anticipated. And the model itself changes: on the cloud road, providers update models, and behaviour you depended on can shift without you doing anything at all. A benchmark that runs weekly catches that in days. No benchmark means you find out when a customer complains.

The ongoing bill, and the levers that cut it

Once you're live, budget for four recurring things: model usage, supporting infrastructure, human review time, and engineering time for retuning and re-benchmarking. The last one is the one that gets forgotten, and it's the difference between a system that stays good and one that slowly stops being trusted.

The good news is that running costs are the most compressible part of the whole picture. Three levers do most of the work:

Model routing. Send each request to the cheapest model that's genuinely good enough. Classification and triage don't need a frontier model; the hard judgement calls do. This alone can halve a bill.

Prompt caching. As the worked example above showed, the repeated portion of your input — instructions, reference documents, retrieved context — can bill at roughly a tenth of the normal rate. Most naive integrations leave this switched off.

Measurement. You cannot cut what you can't see. Log every call with its token counts, latency, and downstream outcome, and review the total against the KPI monthly. When we take over an integration that was built without cost instrumentation, a 40–70% reduction is typical — and it usually doesn't cost any quality, because the spending was in places nobody had ever looked.

Two example budgets

Illustrative only. These are shapes, not quotes — your numbers will differ with scope, data quality, and how much of the work your own team does.

	Cloud API deployment	Regulated on-prem deployment
Year one build	€40k–€90k	€90k–€180k
Model / compute, monthly	€300–€2,000 (usage-based)	€2,000–€6,000 (rented GPUs) or a €250k+ capital purchase
Supporting infrastructure, monthly	€200–€800	€500–€2,000 plus in-house ops time
Human review, monthly	Falls as evals mature	Falls as evals mature
Year two	Retuning, re-benchmarking, scope growth	The same, plus hardware ownership and refresh

The pattern to notice: in year one the build dominates and the running cost is almost noise. By year two, the running costs and the retuning effort are the whole conversation. Budget for both phases, because projects that only fund the build tend to quietly degrade into something nobody trusts.

The one-page cost checklist

Take this to any conversation about an LLM project — with us or anyone else.

• The KPI: what number does this move, and what is it today?
• Token volume: transactions per month × tokens in and out per transaction. Do the arithmetic before you choose a model.
• Model tier: what's the cheapest model that passes your eval set? Have you tested it, or assumed?
• Cloud or on-prem: is there a real regulatory requirement, or an assumed one?
• Break-even: if buying hardware, how many hours a week will it genuinely be busy?
• Data readiness: who's looking at your actual data before the estimate is written?
• Labelling: who produces the known-good answers, and how much of their time does that take?
• Fine-tuning: is it needed, or would a better prompt and retrieval do it?
• Human review: what proportion is reviewed at launch, and what's the plan to reduce it?
• Integration: which systems, whose auth, what happens when one is down?
• Evals: does a benchmark exist, and how often does it run after launch?
• Cost controls: caching, routing, per-user rate limits, hard spending caps.
• The recurring bill: all four buckets — usage, infrastructure, review, engineering.

Any one gap isn't a dealbreaker. A pattern of them means somebody is quoting you the model and hoping the rest works itself out.

How we scope this

This is the whole of what LLM integration services are really buying you — not model access, which you can get with a credit card in four minutes, but the judgement about which road to take and the engineering that makes it survive production. It's the same core underneath our AI agent development and business automation work; we build on Claude for cloud deployments and LLM Studio for on-prem open-weights ones, and we've done enough of both to tell you honestly which one your problem needs.

If you've got a specific use case, book a scoping call. Within a week we'll tell you whether an LLM is the right tool at all, which model to use, what it would cost to build, and what the monthly bill looks like once it's live. If the honest answer is that a spreadsheet and a rule would do the job, that's what you'll hear — and it's a better outcome than a six-figure system that impresses everyone except your accountant.

If you're still earlier than that, our piece on which LLM workflows actually pay back is a good place to start, and the 8-week playbook for a customer-support agent shows what one of these actually looks like from kickoff to live.