Before converting a speculative-decoding speedup into a CometAPI savings estimate, choose the billing unit. That is not a detail to fill in after the performance calculation: it determines which quantities belong in the calculation at all.
CometAPI’s pricing guide separates models with unified official pricing, billed by token, from models without official APIs, billed by number of calls. This category split is the starting point for a route-cost worksheet—not the number of tokens a backend draft model proposes.
Backend quantity versus billing quantity
NVIDIA’s September 2 article on speculative decoding explains two useful quantities: draft length D, the proposed tokens per target iteration, and acceptance length AL, the tokens produced per target iteration. Its speedup calculation balances accepted progress against target verification time and drafting overhead.
In the article’s very-low-latency approximation:
speedup ≈ AL / (1 + ρD)
Here ρ is the draft-model layer count divided by the target-model layer count. Longer drafts help only while acceptance gains outweigh the extra drafting cost. This is an approximation for the described serving regime, not a general API latency guarantee.
Neither AL nor D is a customer billing unit established by the CometAPI guide. Substituting either into a token invoice estimate would confuse internal speculative work with documented billable usage. The NVIDIA article also does not establish that CometAPI deploys this technique or exposes draft-length controls.
Choose the CometAPI billing branch first
For a proposed integration, I would make the tariff category an explicit field in the route configuration and branch the cost calculation accordingly:
| Documented category | Proposed cost calculation | What must be confirmed for the selected route |
|---|---|---|
| Models with unified official pricing: token billing | Sum of billable tokens in each applicable pricing category × its rate per token | Exact usage definitions, applicable token categories, rate units, and account price |
| Models without official APIs: call billing | Sum of billable calls in each applicable call category × its price per call | What counts as a billable call, applicable call categories, and account price |
These are accounting templates, not quoted prices. Convert rates to the same units as the usage before multiplying. The guide was captured on September 5, 2026; confirm the selected model’s current terms before using either branch. Its category description is not proof that a particular route is available, or that the same model can be purchased under both billing methods.
The consequence is concrete. On the token branch, a faster backend with unchanged billable token quantities and rates leaves this modeled charge unchanged. On the call branch, unchanged billable calls and per-call prices likewise leave it unchanged. A speedup may still matter to latency, but dividing an API price by the backend speedup factor is not a supported savings estimate.
A route-selection worksheet
For each task-compatible CometAPI candidate, record the exact model/route, tariff category, price-check date, billable quantities, and total charge. Keep the following evaluation fields separate:
- End-to-end latency: measure it on the intended workload rather than importing NVIDIA’s backend speedup.
- Task acceptance: apply the same quality criterion to each compatible candidate.
- Cost per accepted task: divide total charged spend for the evaluation, including charged unsuccessful attempts, by the number of tasks meeting that criterion. If none qualify, report no qualifying result rather than a finite unit cost.
That is a proposed evaluation, not a reported experiment. It lets a route earn preference through measured latency, quality, and the correct tariff calculation. NVIDIA’s draft-acceptance analysis explains why serving can get faster; CometAPI’s billing-category split determines how to test whether that improvement changes the customer’s economics.