A high token total tells you to investigate a CrewAI run. It does not tell you which agent’s model to replace. In a research → analysis → writing workflow, the apparent cost problem may be repeated completed work rather than an expensive model doing its normal job.
Before changing the routing policy, collect job-linked records for each agent, model and execution attempt—including failures and fallbacks. Keep the final token total as a summary, not the attribution ledger.
This distinction comes from Bobby Spencer’s August 25, 2026 tutorial, “Create Multi-Model CrewAI Agents with CometAPI: A Full Guide”. Its token-tracking and whole-crew fallback examples are useful design evidence, not a benchmark or a verified production deployment.
What the final result cannot settle
The tutorial inspects result.token_usage after run_with_fallback() returns. It explicitly warns that available usage fields depend on the CrewAI version and execution path. Inspect the actual result in your deployed version; do not assume a successful result includes usage from every earlier failed crew attempt.
The fallback example creates another crew and calls kickoff() again when it advances to another route. Consider the failure pattern the article describes:
- Research completes.
- Analysis completes.
- Writing fails.
- A whole-crew fallback runs research and analysis again before reaching the writer.
A research-heavy total could therefore reflect repeated research, not just a costly research task. The same final memo can have a different execution history behind it.
There is another attribution trap: the returned selected_models mapping shows the route used for that crew execution. It is not an attempt history. The tutorial explicitly says its fallback code does not identify the failed agent. A changed writer route is not proof that the writer caused the original failure.
Turn the suggested fields into application records
The article proposes the following usage fields. Treat them as an instrumentation starting point to implement, not as telemetry automatically supplied by the API service or a complete logger already present in the example.
| Fields | Diagnostic purpose |
|---|---|
job_id, agent, model |
Link consumption to a workflow, role and model rather than only the final route. |
input_tokens, output_tokens, total_tokens |
Preserve the usage breakdown when available. |
latency_ms, created_at |
Locate slow execution and place records in time. |
fallback_used, fallback_reason, status |
Distinguish ordinary execution from recovery and failure. |
Its monitoring section also proposes task and start/end timing fields. My recommendation is to add an explicit attempt identifier and retain a separate record for each task execution, so two research executions under one job do not collapse into one row. Record the model actually used on that attempt, not just the final selected mapping.
Capture failure status even when token usage is unavailable. Mark that usage as unknown rather than zero. Otherwise a dashboard can make a failed attempt look free without evidence. These records improve diagnosis; they do not, by themselves, guarantee complete billing reconciliation.
Keep the records metadata-focused. The source warns against logging API keys, private credentials, sensitive prompts, private user data and unredacted model output.
Decide what to change only after attribution
Use the execution records to distinguish three situations:
- Repeated completed tasks dominate: investigate recovery boundaries before swapping models. The tutorial points to checkpoint-based recovery; verify the behavior and configuration for your CrewAI version before relying on it to skip completed work.
- A role consumes heavily even on runs without fallback: evaluate that role’s model assignment against representative tasks. Preserve quality and reliability requirements rather than optimizing token count alone.
- Failed-attempt usage is missing: treat the cost diagnosis as incomplete. Resolve the accounting gap before declaring either the primary or fallback route cheaper.
Finally, tokens are not money. Input/output usage needs applicable model pricing to support monetary attribution; the highest-token agent is not necessarily the most expensive. The tutorial supplies neither measured usage nor an applicable price table or demonstrated savings.
The useful decision is therefore not “Which model has the biggest total?” It is “Are we paying for the intended work, repeated work, or attempts we cannot yet account for?” Answer that first, then evaluate a routing change against cost, output quality and reliability together.