Key Takeaways
- The cost to build an LLM product splits into four lines — scoping, the build, the eval harness, and run cost — and only the second one usually appears in a quote
- Price the build in engineering weeks, not dollars: Prodinit's production engagements land at 4 weeks for a deployment-shaped project, 10–12 weeks for a product-shaped one
- Model tier moves run cost ~20x on identical traffic: 20,000 calls/day costs roughly $4,800/month on Claude Sonnet 5.5 versus roughly $240/month on Haiku 5.5 (Claude pricing)
- The eval harness is the line item teams cut first and regret fastest — without it you cannot tell whether a prompt change improved the product or broke it
- Distillation turns run cost into a one-time engineering cost: a 10-week pipeline cut one platform's inference cost 70% at 10,000 calls/day, saving ~$1,260/month
Most LLM product budgets are wrong in the same direction. They price the thing that is easy to price — a few weeks of application code around a model API — and leave out the three lines that decide whether the product survives contact with real users. The model is rarely the expensive part.
The cost to build an LLM product in 2026 breaks into four components: scoping and data readiness, the build itself, the eval harness, and ongoing run cost. For most teams the build is 50–60% of the first-year spend, and the eval harness plus run cost is the half that gets discovered after launch rather than budgeted before it.
What Drives the Cost to Build an LLM Product
Four cost lines drive every LLM product budget, and each has a different unit of measure. Treating them as one number is why estimates miss: scoping is measured in weeks of discovery, the build in engineering weeks, the eval harness in regression coverage, and run cost in tokens per user per month.
| Budget line | Unit of measure | Typical share of year one | What moves it |
|---|---|---|---|
| Scoping and data readiness | 1–3 weeks | 5–15% | How clean the source data is, how clearly the use case is defined |
| The build | 4–16 engineering weeks | 50–60% | Archetype, integration depth, accuracy target |
| Eval harness | 1–3 engineering weeks | 10–15% | Number of failure modes you must catch before users do |
| Run cost | Tokens/month + infra | 15–30%, recurring | Model tier, call volume, caching, whether you self-host |
The ratio matters more than any single figure. A budget that is 90% build and 10% everything else describes a demo. A budget that reserves real time for data readiness and evals describes a product. Vendor estimates in the $15K–$300K range circulate widely, but they are quoting different archetypes at different accuracy targets — which is why they disagree by 20x and why none of them can price your build without seeing your data.
Cost by Build Archetype: Four Shapes, Four Budgets
LLM products come in four shapes, and the shape sets the budget more than the industry or the model does. The honest unit is engineering weeks, because that is what Prodinit can evidence from delivered work — and weeks convert to whatever blended rate your team or partner charges.
| Archetype | What it is | Engineering weeks | Main cost driver |
|---|---|---|---|
| Model wrapper | Prompt + API call inside an existing product | 2–4 | Integration surface, not AI |
| RAG product | Retrieval over your own corpus, grounded answers | 6–12 | Data pipeline and retrieval quality |
| Agent / workflow platform | Multi-step tool use, orchestration, state | 10–20 | Reliability engineering and failure handling |
| Fine-tuned or distilled system | Custom model trained on your production data | 10–16 | Data curation and eval infrastructure |
Two delivered engagements bracket the realistic range. A deployment-shaped project — taking an existing platform and getting it running correctly on hardened infrastructure — ran 4 weeks, including ten-plus VPC interface endpoints for a fully air-gapped AWS EKS cluster with zero internet egress (air-gapped EKS case study). A product-shaped project is longer: Prodinit's LiveKit voice AI migration for Cuebo took 12 weeks to move real-time voice from a Django WebSocket monolith to a self-hosted LiveKit agent and egress service with five selectable AI pipeline variants, and the resulting infrastructure absorbed 10x peak load.
The jump from wrapper to agent platform is not a jump in model sophistication. It is a jump in how many ways the thing can fail, and therefore how much of the budget goes to handling failure. Agent architecture mistakes are where the 10-to-20-week range comes from.
The Run Cost That Isn't in the Build Quote
Run cost is the line that keeps billing after the build is paid for, and model tier moves it by more than an order of magnitude on identical traffic. Take a product with 1,000 daily active users making 20 LLM calls each — 20,000 calls/day at 2,000 input and 400 output tokens per call, or 40M input and 8M output tokens daily.
| Model | Input / output per 1M tokens | Cost at 20,000 calls/day |
|---|---|---|
| Claude Sonnet 5.5 | $2.00 / $10.00 | |
| Claude Haiku 5.5 (≤100K prompts) | $0.10 / $0.50 |
Model pricing from Claude pricing. Batch processing is priced 50% lower for workloads that tolerate asynchronous turnaround.
That is a ~20x spread from one decision, before any optimization. Three levers compound on top of it: prompt caching on stable system prompts, batch processing for anything not user-facing in real time, and routing by request complexity so the expensive model only sees the requests that need it. Our write-up on LLM cost optimization covers the full set.
The strongest lever converts recurring run cost into a one-time engineering cost. Prodinit built a distillation pipeline — GPT-4.1 teacher to a fine-tuned GPT-4o-mini student, 80,000–100,000 training examples drawn from production call data via Langfuse, deployed through a five-stage progressive A/B rollout — that cut per-call inference cost from roughly $0.006 to $0.0018 for a platform handling 10,000 calls/day, a 70% reduction worth about $1,260/month at that volume (distillation case study). The engagement took 10 weeks. Below a few thousand calls a day that trade is not worth making; above roughly 10,000 calls/day it pays back inside a year and keeps paying as volume grows. What fine-tuning actually costs breaks that threshold down further.
Where LLM Product Budgets Actually Break
Budgets rarely break on the model bill. They break on four predictable items that look optional during scoping and turn out to be load-bearing: data readiness, the eval harness, the last stretch of accuracy, and retrieval quality once the corpus is real rather than sampled.
Data readiness. The build cannot start until the source data is reachable, clean, and permissioned. On a clinical trial analytics engagement, the dashboard layer was straightforward; the work was getting live PostgreSQL data into a shape a natural-language-to-SQL layer could query safely, which took reporting latency from 1–2 days to real time (clinical trial dashboard case study). Teams that budget zero weeks here spend them anyway, unbudgeted.
The eval harness. This is the first line cut and the fastest regretted. Without regression evals you cannot answer whether last week's prompt change improved the product or quietly broke a case that used to work, so every release becomes a guess. Budget 1–3 engineering weeks and treat it as a deployment gate, not a report — how to build evals that catch regressions covers the mechanics.
The last 10% of accuracy. Getting an LLM feature to roughly 80% correct is fast. Taking it from 80% to the 95%+ that a user-facing product needs frequently costs as much as everything before it, because the remaining failures are long-tail and each needs its own fix. This is the single most common reason an LLM project doubles in cost: the accuracy target was never stated, so the build was scoped against the demo.
Retrieval quality at real scale. RAG that works on a 200-document sample behaves differently on 200,000 documents — chunk boundaries, stale embeddings, and missing metadata filters all surface only at volume. Why RAG pipelines fail in production catalogues the failure modes worth budgeting for in advance.
Build In-House, Contract, or Partner
The delivery model changes the cost structure more than the total. Hiring builds permanent capacity at a permanent cost and a hiring lag; contracting buys hands without architecture ownership; partnering buys a delivered outcome on a fixed scope. The right answer depends on whether LLM engineering is core to your product long-term.
| Situation | Recommended path |
|---|---|
| LLM work is core and continuous; you can absorb 3–6 months of hiring lag | Build in-house — see how to hire AI engineers |
| First LLM product, no in-house ML infra experience, needs to ship this quarter | Partner — the eval and LLMOps scaffolding is where a specialist saves the most time |
| Clear spec, existing senior AI engineer to own architecture | Contract — hands scale a plan someone internal already owns |
| Regulated or air-gapped deployment target | Partner — compliance infrastructure is front-loaded and expensive to learn once |
| Exploring whether the use case is viable at all | Scope first — a 1–3 week discovery is cheaper than a mis-scoped 12-week build |
The question that resolves most of these: is the accuracy target written down, and does someone own it? If yes, almost any delivery model works. If not, no delivery model saves the budget — estimating AI timelines honestly starts with that number, and so does estimating cost.
Get Prodinit's AI engineering guides in your inbox
Deep-dives on production LLMs, voice AI, and MLOps — published weekly. No sales emails.
Frequently Asked Questions
It depends on archetype. A model wrapper inside an existing product is 2–4 engineering weeks; a RAG product is 6–12; an agent platform is 10–20; a fine-tuned system is 10–16. Multiply by your team's or partner's blended rate, then add 15–30% of year-one spend for recurring run cost. Vendor-quoted dollar ranges span $15K–$300K+ because they quote different archetypes at different accuracy targets.
The gap between a demo and a product — specifically the eval harness and the last stretch of accuracy. Reaching roughly 80% correct is quick; reaching the 95%+ a user-facing product needs often costs as much again, because the remaining failures are long-tail and individually diagnosed. Budgets that never stated an accuracy target are the ones that double.
For most products, an API is cheaper until volume or compliance forces the change. API pricing varies ~20x by model tier alone — roughly $4,800/month versus $240/month at 20,000 calls/day on Claude Sonnet 5.5 against Haiku 5.5 (Claude pricing). Self-hosting makes sense when you need air-gapped deployment, data residency control, or volume high enough that GPU amortisation beats per-token pricing.
Four levers, in order of effort: route requests by complexity so the expensive model only handles hard ones, cache stable system prompts, batch anything not real-time for roughly half price, then distill a smaller model on your own production traffic. Prodinit's distillation work cut one platform's inference cost 70% at 10,000 calls/day with no quality regression, measured through staged A/B gates.