LLM Cost Management: Control Spend by Running Less AI
LLM cost management starts with visibility, budgets, routing, and attribution. The durable fix is architectural: move repeatable work into deterministic apps so token demand grows more slowly as usage grows.

LLM cost management is the practice of metering, attributing, budgeting and reducing what a system spends on model inference and everything attached to it: input and output tokens, retries, tool calls the model triggers, retrieval infrastructure and the human time spent catching bad output. Metering and budgets make spend visible. Only moving repeatable work out of the model makes it smaller, and that is the lever most cost guides skip.
If you run platform, finance or AI adoption for a company with more than a few agents in production, this article gives you the controls to put in place first, then the structural change that bends the curve. On Major, the structural change is the product: agents build deterministic apps for the repeatable parts of a workflow, so the model is billed for judgment and not for lookups.
What does LLM cost management actually cover?
Write the spend out and the lever becomes visible.
cost = calls x (1 + retry rate) x (input tokens x input price + output tokens x output price) + tool fees + retrieval infrastructure + review time
Most cost work targets the two price terms. Some targets input tokens. Almost none targets calls, the only term that grows with adoption. Agentic workloads made that term worse: one user request now fans out into planning steps, tool selections, retries on malformed output and verification passes, inside code most FinOps owners cannot see into.
Instrumentation is a prerequisite for reduction and does not reduce anything by itself. A team can ship perfect per-tenant attribution and spend exactly as much next month.
What should you meter on every model call?
Emit one event per call, including failed calls, because a failed call that consumed input tokens still costs money and a collector that records only successes under-reports exactly when the system misbehaves.
Carry these fields: timestamp, tenant_id, workflow_id, step_id, model, input_tokens, output_tokens, cached_input_tokens, attempt, outcome, cost_usd, price_version, prompt_hash and the credential that authorized the call. Store the hash of the prompt and not the text, so repeated-work analysis does not create a second copy of customer data. Scope the cost store by tenant the way you scope the application database. Keep price_version so you can reprice history when a price sheet changes; hardcoded prices are the most common reason a cost pipeline goes quietly wrong.
How do you attribute LLM costs to teams and use cases?
Attribute to workflow and step, not just to team or project. "The support agent cost $40k last quarter" cannot be acted on. "The ticket-classification step cost $40k last quarter" can. The FinOps Foundation's framework ties accurate forecasting to "the ability to fully categorize and allocate technology costs" (FinOps Framework), and an unallocated pool of model spend cannot be forecast, defended or cut.
Name one person per workflow as owner. A person gets the alert; a team shares it.
How do you set an LLM budget?
Baseline 30 days of per-workflow spend, then set a soft threshold near 80 percent of that baseline and a hard stop at the ceiling. A real budget has four fields: soft threshold, hard stop, escalation path and named owner. Anything less is a chart.
Add a per-run token ceiling. Runaway retry loops do their damage inside hours, and a monthly check will not catch a loop that burns a quarter's allowance overnight. Every hard stop also needs an exception path that writes an audit record. Without one, the first production incident gets resolved by someone turning the limit off for good.
Which per-call levers help, and where do they stop?
Three levers lower the price of a call. None removes the call.
- Prompt caching. Anthropic bills cache reads at "0.1 times the base input tokens price," with 5-minute writes at "1.25 times" and 1-hour writes at "2 times" (Anthropic prompt caching docs). A prefix written and never re-read costs more than skipping the cache. Put stable instructions first and volatile input last.
- Batching. Anthropic's Message Batches API describes "reducing costs by 50%," with most batches completing within an hour and a 24-hour expiry (batch processing docs). Plan a re-submission path for expired requests.
- Routing. Sending cheap tasks to cheap models pays when difficulty varies and classification is cheap. It stops helping when the classifier is itself a model call, when a downgrade raises the retry rate enough to cancel the saving, or when a task depends on one model's tool-use behavior.
Take all three. Then notice what they share. An agent that re-derives the same customer-tier classification on every run gets a cheaper re-derivation, then does it again tomorrow.
Which steps should leave the model?
Take a collections workflow. The agent pulls an invoice, checks it against contract terms, decides whether payment is late, drafts a note, updates the system of record and escalates disputes. Five of those six steps produce a stable output for a given input: lookups, comparisons, format changes, writes. One needs a model: whether a disputed case should escalate.
Run all six through the model and you pay for all six on every invoice. You also get no durable state, because the intermediate results lived in a context window that ended with the run. The cost problem and the state problem are the same problem. Work that is not stored gets redone, and redone work is billed again.
Use this test per step:
- Does it produce the same output for the same input? If yes, it is code.
- Are the rules still changing week to week? If yes, keep it in the model until they settle.
- Does it need judgment about ambiguous input, or only a lookup, comparison, transformation or write?
- Can you name the owner, permissions and audit record for it today? If not, moving it into code gives you all three.
How to build this on Major
Major is the enterprise platform where agents build the software they run on. The cost argument above becomes a build, not a policy:
- Cost ledger app. An app with a managed database holds one row per model call with the fields above, plus views for cost by workflow, step and owner. It is the control surface finance and platform leads open.
- Warehouse and billing connectors. Pull provider usage exports into the ledger through your warehouse connector (Snowflake, BigQuery or Postgres) so provider invoices reconcile against metered events.
- A cost agent that judges, then builds. The agent reviews the ledger for repeated work using
prompt_hash, picks out steps that pass the four-question test, and builds an app for each one. Its judgment call is which steps are stable enough to compile, and it escalates the rest. - Scheduled workflow. A daily workflow runs the budget checks as deterministic code and posts breaches to Slack, with the named owner tagged. No model call is needed to compare a number to a threshold.
- Governance at the point of action. Credentials are scoped per app, access is role-based, and every run writes an audit record, so an exception to a hard stop is attributable.
Start building free and have the first ledger app running the same day.
Where this approach does not fit
An app built from a wrong understanding runs the wrong logic reliably, which is worse than an inconsistent model getting it right half the time. The permissions, logs and review path around the app matter as much as the app. A workflow whose rules change weekly should stay in the model, because you will pay to rebuild the app more often than you save by running it.
The cost shape is front-loaded and then flatter: building the app costs reasoning up front, and running it costs what code costs. We will not put a percentage on that, because the size of the effect depends on how much of your workflow is genuinely repeatable. This article also skips self-hosted GPU economics, fine-tuning spend and evaluation cost. Evaluation deserves its own treatment, since the quality bar you enforce sets the retry rate that sets the bill.
Reason once. Run forever.
Start building on Major
Put the cost ledger, the budget workflow and the compiled steps on one governed platform, so every dollar of model spend has an owner and every repeatable step stops being billed twice.
Related articles
Frequently asked questions
- What is LLM cost management?
- LLM cost management is the practice of metering, attributing, budgeting, and reducing what a system spends on model inference and its dependencies. That covers input and output tokens, retries and re-runs, tool calls the model triggers, retrieval infrastructure, and human review time. Metering and attribution make the spend visible. Reducing it means changing where the work runs.
- How do companies control LLM costs?
- They emit one cost event per model call, attribute spend to a named owner per workflow and step, and enforce graduated budgets that end in a hard stop. Then they lower the price of each call through prompt caching, batch APIs, and routing cheap tasks to cheap models. The larger lever is removing repeatable steps from the model entirely and running them as deterministic code.
- How do you set an LLM budget?
- Baseline 30 days of per-workflow token spend, then set a soft alert around 80 percent of that baseline and a hard stop at the ceiling. Add a per-run token ceiling, because retry loops burn a monthly allowance overnight. Name one person as owner rather than a team, and define an exception path that writes an audit record when someone raises a limit.
- Does model routing reduce LLM costs?
- Routing reduces cost when task difficulty varies widely and you can classify it cheaply. It stops helping when the classifier is itself a model call, when the cheaper model raises the retry rate enough to cancel the per-token saving, or when the task depends on a specific model tool-use behavior. Routing lowers the price of a call and does not remove the call.