Engineering Teams Don’t Buy Tokens. They Buy Accepted Outcomes.

The cheapest model is not necessarily the cheapest way to get useful work done. Production AI economics should be measured around successful work, not inference alone.

For the past few years, one of the most common questions in AI engineering has been deceptively simple: how much does this model cost per million tokens? Teams compare input and output rates, cached-token discounts, batch pricing, and the difference between larger and smaller models before deciding which provider or model belongs in production.

That analysis is useful, but it measures only one part of the system. A business does not ultimately purchase tokens. It purchases work: a bug fixed correctly, a pull request that survives review, a customer case that is actually resolved, a report that an analyst is willing to use, or a workflow that completes without creating more work downstream.

Once AI systems begin performing bounded jobs rather than simply generating content, the more useful economic unit becomes cost per accepted outcome.

I would define it as:

Cost per Accepted Outcome = Total Workflow Cost / Accepted Outcomes

The important phrase is total workflow cost. It includes model inference, but also retrieval, tools, searches, execution infrastructure, retries, human review, rework, failures, and other costs required to turn a model response into something the organization considers complete.

That distinction changes almost every conversation about AI optimization.

Token cost is infrastructure cost, not business value

Current model pricing already hints at how incomplete a token-only view has become. OpenAI's current GPT-5.6 family spans Sol, Terra, and Luna with materially different input and output prices, while cached input is priced separately and eligible batch processing is offered at a discount. Gemini pricing goes further by exposing standard, batch, flex, and priority processing, with separate economics for context caching and grounding through Google Search or Maps.

These pricing structures are useful because they allow engineering teams to optimize infrastructure consumption. A high-volume asynchronous workload may benefit from batch processing. A long-lived context may benefit from caching. A latency-sensitive interaction may justify premium processing. A grounded research task may incur search costs that do not exist in a simple extraction workflow.

However, none of these numbers tells us whether the result was useful.

An engineering team could reduce model spend by 40% and still make the overall workflow more expensive if the cheaper configuration produces more incorrect patches, triggers more retries, or increases the amount of senior-engineer review required. Conversely, spending significantly more on a stronger model may reduce total cost if it increases the probability of producing acceptable work on the first attempt.

Token pricing therefore belongs in the numerator of the economics equation. It should not define the equation itself.

A cheap attempt can create an expensive outcome

Consider a simplified coding example. Assume Model A costs only $0.10 in AI and tool usage per bug-fix attempt. It looks extremely inexpensive, but the output often needs correction, so the expected human review and rework cost is around $4 per attempt. If only 45% of attempts ultimately meet the team's acceptance criteria without requiring another full execution, the workflow cost per attempt is approximately $4.10 and the effective cost per accepted result is roughly $9.11.

Now consider Model B. It costs $0.70 in AI and tool usage per attempt, seven times more than Model A at the infrastructure layer. However, it requires only about $1.50 of review and rework, while 80% of attempts satisfy the acceptance criteria. The workflow cost per attempt becomes $2.20, which produces a cost per accepted outcome of $2.75.

The numbers are hypothetical, but the mechanism is not. Model B looks dramatically more expensive if the engineering team compares only API prices, yet it is more than three times cheaper when the unit of analysis becomes completed work.

This pattern is familiar throughout software engineering. A cheaper database is not cheaper if it creates large operational overhead. Faster feature delivery is not economically efficient if it creates a stream of incidents and rework. A low-cost vendor is not necessarily cheaper if internal employees spend significant time correcting the output.

AI systems should be evaluated through the same systems lens.

The denominator is where the business value lives

Most cost dashboards focus naturally on the numerator because spend is easy to measure. We can count tokens, API calls, search requests, GPU time, and dollars consumed by each application or team. These metrics are valuable for budgeting and operational control, but they become misleading when they are presented as productivity metrics.

Suppose one engineering organization spends $10,000 per month on AI while another spends $25,000. The first organization appears more efficient until we learn that its AI systems generate 500 accepted engineering outcomes while the second produces 3,000. The first organization is effectively spending $20 per accepted outcome, while the second is spending approximately $8.33.

The larger AI bill is producing better unit economics.

This is why AI FinOps will eventually need to move beyond answering where the money went toward answering what the organization received for that money. Cost attribution by team, model, and provider is useful, but cost attribution by workflow and successful outcome is much closer to the information engineering and business leaders actually need.

A statement such as “the platform consumed 800 million tokens this month” tells us how much infrastructure was used. A statement such as “the platform helped complete 1,800 accepted code changes at an average AI-enabled workflow cost of $6.20 each” tells us something about productivity.

The denominator converts consumption into economics.

An accepted outcome must represent usable work

Defining the denominator is the hardest part because “accepted” must mean more than “the model returned an answer.” Generation is not completion, and a syntactically valid response is not necessarily useful work.

For a coding workflow, an accepted outcome might require that tests pass, CI remains green, the change satisfies the ticket's acceptance criteria, and a reviewer finds no critical issue before merge. For a support agent, acceptance may mean that the response follows policy, resolves the customer's problem, and does not lead to a reopened case. For an incident agent, it could mean the root cause was correctly identified, remediation succeeded, and the incident owner accepted the resolution.

The definition should therefore emerge from the workflow, not from the model.

This is consistent with how agent evaluation itself is evolving. Anthropic's guidance on agent evals emphasizes that a meaningful evaluation increasingly needs to test the complete multi-step task, including tool use and changes to the environment, rather than grade only a single response. A coding agent, for example, can be evaluated by whether its implementation actually passes unit tests after completing the agent loop.

That is an important conceptual shift. Instead of asking whether the model generated a convincing answer, we ask whether the system successfully performed the job.

Accepted outcome is essentially the economic counterpart of that evaluation philosophy.

Acceptance rate becomes a critical production metric

Once an organization can define acceptable work, one of the most useful operational metrics becomes:

Acceptance Rate = Accepted Outcomes / Attempts

This metric exposes problems that token dashboards often hide. Imagine that a routing change reduces average token consumption by 30%, which looks like a strong optimization. If acceptance rate simultaneously falls from 82% to 61%, the organization may simply have moved cost from inference into retries and human rework.

The reverse can also happen. Moving a workflow to a more capable and more expensive model might increase inference cost by 40%, yet raise acceptance from 65% to 90% and substantially reduce review time. That can improve economics despite the visibly larger vendor bill.

I would therefore evaluate production AI using at least four connected measures: cost per attempt, acceptance rate, cost per accepted outcome, and time per accepted outcome. These four numbers reveal the tradeoff among spending, reliability, and cycle time much better than token consumption alone.

They also provide a cleaner basis for comparing different system designs. A team can compare two models, two prompting strategies, or two agent architectures in terms of the work they actually produce rather than the resources they consume internally.

Human review belongs inside AI economics

Human review is likely to be one of the most underestimated costs in agentic systems because it does not appear on a provider invoice. If a workflow costs $0.20 in inference but requires ten minutes of senior-engineer review, calling it a twenty-cent task is economically meaningless.

Assume a fully loaded engineering cost of $100 per hour. Ten minutes of review represents roughly $16.67 of labor. Reducing inference from $0.20 to $0.10 saves ten cents, while reducing review from ten minutes to five saves more than eight dollars. The second optimization is obviously more valuable even though it may not appear anywhere in the AI spending dashboard.

Human involvement can also take many forms. Reviewers may validate facts, correct outputs, re-run failed workflows, refine prompts, debug agent behavior, resolve escalations, or repair downstream consequences. All of this is part of the system's cost of producing acceptable work.

This does not imply that human review is undesirable. In many high-risk workflows it is exactly what makes the system safe enough to operate. The economic question is whether the level of review matches the value and risk of the task.

A five-minute approval before a large financial transaction may be excellent economics. Five minutes of review on every routine document classification probably indicates that the automation boundary has been designed poorly.

Retries are hidden workflow waste

Retries create a similar measurement problem. A user may see only one successful outcome, while the underlying agent has attempted the task several times because earlier runs chose the wrong tool, produced invalid structured output, lost necessary context, hit an API failure, or followed an execution path that eventually failed.

From a product perspective, the workflow succeeded once. From an economics perspective, the organization paid for every failed path that came before it.

This makes Attempts per Accepted Outcome another useful metric. If that number increases over time, the problem may be model quality, tool reliability, orchestration complexity, badly chosen retry policies, or changes in the distribution of tasks entering the system.

The metric is particularly valuable for agents because they can hide significant internal work behind a simple user interface. An apparently inexpensive agent call may contain multiple model turns, searches, database queries, code executions, sub-agent calls, and retries before producing the result that the user eventually sees.

The cheapest retry is the one the architecture did not need.

Model routing should optimize expected outcome economics

Most model-routing strategies begin with a simple heuristic: easy tasks go to cheaper models, difficult tasks go to stronger models. That is a reasonable starting point, but a mature system should optimize something more useful than model price.

The relevant question is:

Which execution strategy gives us the lowest expected cost for an acceptable outcome?

An execution strategy can include the selected model, reasoning effort, context size, retrieval strategy, available tools, processing tier, retry policy, and human-review threshold. Current pricing systems already expose many of these variables. OpenAI's GPT-5.6 family offers substantially different price points across Luna, Terra, and Sol, while Google's Gemini platform differentiates standard, batch, flex, and priority processing and prices caching and grounding separately.

The cheapest model can therefore lose even before quality is considered. A task requiring multiple retries on a small model may be cheaper if it goes directly to a stronger model. A high-latency batch workflow may benefit enormously from discounted asynchronous execution, while an interactive customer workflow may justify a more expensive low-latency tier.

The router of the future is not simply choosing the cheapest adequate model.

It is choosing an economic operating point for the workflow.

Public benchmarks cannot tell you your cost per accepted outcome

This framework also explains why public benchmarks should not determine production model selection on their own. A benchmark can reveal useful information about general capability, but it cannot reproduce the exact repositories, policies, tools, customer behavior, approval thresholds, or failure costs inside a particular organization.

Two models with similar benchmark scores may have very different economics in a real workflow. One may use more tools but require less human review. Another may perform well on average but fail disproportionately on a category of requests that is unusually common in the company's workload. A coding model may score highly on a public benchmark while interacting poorly with a specific monorepo or internal framework.

The only reliable way to answer these questions is to evaluate candidate configurations against representative internal tasks.

This is another reason production evals and AI FinOps are beginning to converge. The eval suite tells us whether work is acceptable; the cost instrumentation tells us what it took to produce that acceptable result. Without the first, we can optimize cost while quietly destroying quality. Without the second, we can improve quality without understanding whether the economics remain sustainable.

Together they create a real unit-economics model for AI work.

Measure the workflow before optimizing the model

For engineering leaders, a practical implementation can begin with a very small scope. Choose one bounded workflow rather than measuring “AI productivity” across an entire organization. A useful example might be resolving small production bugs from triage through a tested patch.

Next define exactly what counts as accepted. The patch might need to satisfy the ticket, pass CI, contain no critical review issue, and be merged. Then instrument the entire path: attempts, accepted outcomes, model consumption, tools, searches, execution costs, retry count, elapsed time, review time, escalations, and failures.

Only then should the team begin optimizing.

The largest cost may turn out to be output tokens. It might instead be web search, long contexts, unnecessary reasoning, excessive retries, expensive sandbox execution, or human review. A supposedly cheap model may be the main source of rework, while a more expensive model may improve the entire workflow enough to reduce total cost.

This is simply performance engineering applied to AI economics. We would not optimize a distributed production system by staring at the price of one CPU instruction. We instrument the system, locate the bottleneck, and optimize the component that moves the overall result.

AI should be treated with the same discipline.

Cost per accepted outcome connects engineering metrics to business metrics

One reason I find this framework useful is that it gives engineering and business teams a shared unit of discussion.

Engineering can still reason about models, caching, token usage, latency, retries, and tool design. Finance can reason about total cost. Product and operations can define what counts as completed work. Management can compare the resulting unit economics with the previous human or software workflow.

Suppose an AI-assisted support workflow costs $4 per accepted resolution. That number is difficult to interpret in isolation. If the previous process costs $18 per resolution, the economics are excellent. If a conventional rules engine can resolve the same cases for $0.40, AI may not be the right tool.

The same logic applies to software engineering. An AI coding workflow costing $12 per accepted fix could be highly valuable if the equivalent engineering effort previously cost $150. Another AI workflow costing only $0.50 may be wasteful if employees spend ten dollars reviewing each result.

Cheap AI is not automatically valuable AI.

Value comes from the relationship between cost and the unit of useful work.

This is where AI FinOps becomes different from cloud FinOps

Traditional cloud FinOps focuses heavily on allocating infrastructure consumption, identifying waste, forecasting spend, choosing purchasing models, and balancing cost against performance. AI needs all of those disciplines, but agentic software introduces another challenge because the amount of compute consumed is only loosely related to the amount of useful work produced.

Two executions of the same agent can follow completely different paths. One may solve the problem in three model calls while another searches repeatedly, invokes several tools, retries a failed operation, escalates to a stronger model, and eventually requires human intervention.

The infrastructure bill captures all of this activity. It does not explain whether the activity created value.

AI FinOps therefore needs to connect cost telemetry to workflow telemetry. Instead of only asking how much a team spent on GPT-5.6 Sol or Gemini 3.5 Flash, organizations should eventually be able to ask how much they spent resolving payroll cases, completing engineering changes, performing compliance reviews, or generating research that was accepted by its users.

That is when AI cost management becomes AI unit economics.

The engineering-management objective needs to change

This framework also changes the incentives engineering leaders set for teams.

An objective such as “reduce AI spend by 20%” is easy to understand but can produce counterproductive optimization. Engineers may route more traffic to weaker models, reduce useful context, disable expensive tools, or shift work back to humans. The AI invoice improves while the real workflow becomes slower or more expensive.

A stronger objective would be:

Reduce cost per accepted engineering outcome by 20% without degrading quality or cycle time.

Now the team is free to optimize the entire system. It might choose a cheaper model, but it could also use a stronger one. It might introduce caching, improve tool design, reduce retries, automate validation, alter the review threshold, or remove AI from a step where the economics are poor.

This is a healthier engineering goal because it is aligned with the outcome rather than with one vendor's invoice.

It also creates a clearer way to evaluate productivity claims. If AI spending doubles while cost per accepted result falls substantially, the system may be improving. If token consumption falls but cost per accepted outcome rises, the organization has optimized the wrong thing.

AI is becoming economically measurable work

The broader reason this matters is that agentic AI is moving model consumption closer to recognizable units of work. A generic chatbot conversation can be difficult to value. A bounded agent workflow is much easier to measure because it has a beginning, an execution path, and an outcome that the organization can accept or reject.

As that transition continues, we should expect AI economics to become much more operational. Teams will track cost per resolved case, cost per accepted code change, cost per completed research task, and cost per approved workflow rather than viewing model spend only through aggregate token dashboards.

The infrastructure optimizations will still matter. Caching, batch processing, model routing, prompt design, context management, and provider pricing can make substantial differences to cost. OpenAI and Google already expose enough pricing dimensions to make those choices meaningful at scale.

But those techniques are subordinate to a larger question.

The fundamental objective is not to minimize the cost of generating intelligence.

It is to minimize the cost of converting intelligence into useful work at the quality, latency, and risk level the business requires.

That is the distinction engineering teams need to preserve as AI systems become more autonomous.

Because engineering teams do not ultimately buy tokens.

They buy bugs fixed, pull requests accepted, incidents resolved, decisions improved, and workflows completed.

They buy outcomes they are willing to accept.