# Model-route cost comparison Compare routes on the same eligible work, permitted inputs, acceptance rule, period, and quality limits. A smaller model bill alone does not establish lower workflow cost. A route is the whole candidate setup: its model, prompts, tools, checking, and recovery. Start by agreeing which tickets both routes must handle and how the reviewer will accept or reject a draft. Then price all the work those tickets create, including rejected and unfinished cases. This worksheet contains a completed hypothetical calculation and a blank record. Its USD amounts, counts, and minutes are authored teaching assumptions. They are not measured performance, provider prices, cash savings, or release evidence. ## Completed hypothetical support example - Period: one month, through a declared reporting cutoff. - Eligible and reached population: the same 1,000 unique support tickets for each route. - Scope: draft issue category and urgency with ticket evidence; no customer contact, account change, or ticket closure. - Accepted unit: independently checked classification draft with a recorded final disposition, including corrected drafts that subsequently pass the same acceptance rule. - Each route: 900 accepted, 60 rejected, 40 escalated and unresolved at cutoff. - Review: all checking and correction minutes incurred across all 1,000 reached cases by cutoff. Include rejected and escalated work. - Loaded review rate: USD 45/hour = USD 0.75/minute, a planning assumption. | Cost item | Route A / route B, USD | | ------------------------------- | ----------------------------------------- | | Initial model calls and retries | 20 / 200 | | Tools, compute, storage | 80 / 80 | | Review and correction | 2,500 × 0.75 = 1,875 / 1,200 × 0.75 = 900 | | Separate technical recovery | 120 / 60 | | Routine support allocation | 200 / 200 | | Assurance allocation | 100 / 100 | | Delivery allocation | 200 / 200 | | Total allocated service cost | 2,595 / 1,740 | | Accepted results | 900 / 900 | | Cost per accepted result | 2,595 / 900 ≈ 2.88 / 1,740 / 900 ≈ 1.93 | Delivery assumes USD 2,400 of one-time setup allocated over 12 months. Record actual cash timing separately and never add both the full payment and allocation to one period's cost. Recovery is technical diagnosis/repair outside routine support; review is ticket checking/correction. Assurance is the work of checking that the service continues to meet its requirements. These categories are planning assumptions, not reports of completed work. Prevent overlapping labor charges. Costs include the 100 unaccepted cases through the cutoff. Their remaining resolution cost is unknown. This is not a complete ticket-lifecycle cost estimate; follow their backlog and later work into subsequent periods. Other omitted integration, licensing, incident, or waiting costs must be added if applicable. ```text Additional model cost for B = 200 − 20 = USD 180 Lower review cost for B = 1,875 − 900 = USD 975 Lower separate recovery cost for B = 120 − 60 = USD 60 Difference in allocated cost = 975 + 60 − 180 = USD 855 A's cost outside review = 20 + 80 + 120 + 200 + 100 + 200 = USD 720 A's break-even review cost = 1,740 − 720 = USD 1,020 A's break-even review minutes = 1,020 / 0.75 = 1,360 At 1,000 review minutes, A costs 720 + 1,000 × 0.75 = USD 1,470 ``` The break-even holds B's USD 1,740 cost, both 900 accepted counts, and all other original inputs fixed. It does not prove how to attain the lower review effort. Do not remove required verification to reach it. In a separate denominator stress case, hold B's cost at USD 1,740 and assume 600 accepted drafts: USD 2.90 each. Holding cost fixed is an arithmetic sensitivity, not a forecast that failed cases need no additional effort. With zero accepted drafts, cost per accepted draft is undefined; report cost and count separately. ## Gates and missing evidence Both routes remain unqualified for production. There is no measured urgent-case miss rate, disclosure result, latency, review-burst capacity, residual loss, manual baseline, or sustained result. The 900 accepted counts are assumed and establish none of those properties. Before a real comparison, the owner must define critical error slices, permitted source and destination use, acceptance authority, latency and queue limits, overflow, fallback, and stop conditions. A price advantage cannot offset a failed safety requirement. Freeze inputs, versions, resource limits, cases, and evaluation conditions; repeat stochastic runs where necessary. Measure active effort separately from browser elapsed time. ## Blank matched-route record Begin with the shared job, counts, and checking time. Leave unknown costs visible instead of treating them as zero. The remaining questions make the comparison repeatable and show what prevents a production choice. ### Start with these questions 1. Which tickets and evidence will you give both routes? 2. What must the reviewer accept, and how many cases remain rejected or unresolved? 3. How much checking and correction does each route create across all tickets? 4. Which required safety or quality check would rule out a cheaper route? ### Establish a matched comparison - What bounded decision does the workflow support, and which actions are outside scope? - Who owns the workflow and service, and who independently checks the result? - Which period, cutoff, currency, and cost basis apply? When are payments actually made? - Which unique tickets may both routes handle, which are excluded, which actually reach each route, and how are retries attached to the same ticket? - Which source evidence was available when each request was made? What access and destination limits must both routes respect? - What counts as one accepted result, and who approves the reference answer? - How many cases were accepted, rejected, pending, or escalated for each route at cutoff? - Which critical-error, privacy, reliability, latency, and review-capacity requirements must pass? - Which model, prompt, context, tools, runtime, resource limits, and checker versions define route A? - Which equivalent versions define route B, and exactly what differs? - How much work does today's manual process take, and how will you give the candidates comparable tickets and evidence? - If answers vary between runs, how will you repeat the tests and report that variation rather than selecting the best attempt? - What did initial, failed, and retried model requests cost? - What did retrieval, tools, compute, storage, and any priced waiting cost? - How much review and correction work was done across all reached cases, at which labor rate? - Which technical recovery, routine support, and assurance costs are included? How are their boundaries kept separate? - Which delivery/change costs are allocated, which licenses and later obligations apply, and what is the payment schedule? - Which costs were incurred through cutoff, and which unresolved backlog or future costs remain unknown? - What are the full cost and independently accepted count for each route? - What is cost per accepted result? If the accepted count is zero, leave the ratio undefined and report cost and count separately. - How much would checking time or the accepted count need to change to reverse the choice? Which assumptions stay fixed, and what if several worsen together? - Required checks that must not be removed to lower cost: - Missing evidence, assigned owner, and next review point: - Decision: compare further, constrain, retain current route, pause, or select within approved scope: ## Optional spend diagnosis For model-mediated work, inspect users × sessions/user × turns/session × requests/turn × tokens/request × price/token. Use consistent units; split mixed models and input/output/cached prices into separate classes, then reconcile them to the bill. A changed factor can locate an investigation but does not prove cause or replace full cost per accepted outcome. When isolating a context, cache, or batching change, hold model, representative workload, and enforced budget constant. Test a model replacement separately. Preserve failed, inconclusive, and nonaccepted work.