Switching CrewAI calls to acall is a dispatch change, not evidence that your CometAPI route has spare capacity. Before starting more requests, put an application-owned admission boundary around the calls that share that gateway configuration.
CrewAI’s versioned LLM documentation documents asynchronous acall and says it accepts the same parameters as synchronous call, including messages, tools and callbacks. It also recommends rate limiting. Neither statement gives a safe parallelism setting for your deployed route.
The relevant integration detail is in CometAPI’s CrewAI guide: agents use explicit model settings through https://api.cometapi.com/v1, and its example shares the CometAPI key and base URL across an agent-to-model map. That is the configuration boundary to account for when independent calls become concurrent—not a fresh capacity allowance for each agent.
Put admission before dispatch
I would use this application policy before increasing concurrency:
- Keep task dependencies intact. Parallelize independent calls, not a downstream task that still needs an upstream result.
- Bound pending work as well as active calls. A semaphore around requests is not enough if producers can create an unlimited backlog. Choose a queue limit and an explicit overflow outcome, such as deferring the job or returning a busy response.
- Coordinate aggregate and per-model budgets. Admit a request only when both the shared gateway budget and its model budget allow it. A single dispatcher can make that decision together, without reserving a global slot while waiting indefinitely for a model slot. Separate model queues must not be treated as separate provider allowances.
- Include every worker sharing that configuration. A local limit in each process does not bound their combined demand. Use shared admission coordination or explicitly partition the application budget across workers.
- Release admission capacity on completion, error and cancellation. Keep the existing retry owner; this change should not introduce a second retry loop.
These are application design recommendations, not claims that CrewAI or CometAPI supplies this controller. An in-flight cap also does not enforce a requests-per-minute allowance: faster completions can increase request rate at the same concurrency. Where a rate budget is known, enforce it separately.
Separate waiting from service time
CometAPI’s guide recommends records with job, agent, model, latency and status, plus per-model latency percentiles and reliability metrics. Extend those application records with queue-entry, dispatch and completion timestamps:
| Signal | What it helps decide |
|---|---|
| Pending count and queue wait | Whether the application is accumulating work before dispatch |
| In-flight count, aggregate and by model | Whether the configured admission budgets are actually respected |
| Dispatch-to-completion latency, including p95 | Whether admitted requests are slowing down |
| Status, timeouts and 429 counts, aggregate and by model | Whether increasing demand coincides with rate limiting or reliability degradation |
These are fields to collect in your application, not a promised CometAPI response schema. Keep API keys, sensitive prompts, private user data and unredacted model output out of these records, as the guide cautions.
Increase only against a stop condition
Start with a deliberately small application budget and a representative workload. Define acceptable queue wait, service latency and error thresholds before changing admission. Increase in small steps only while those observations remain within budget; neither source supplies a universally safe count.
If queue wait grows while service latency remains stable, inspect offered load and the application cap before deciding whether to admit more. Queue wait alone is not proof of spare gateway capacity. If service latency or timeouts worsen, hold or reduce admission rather than trying to drain the queue by launching more work.
For 429, the CometAPI guide says to back off. Stop raising admission and apply backoff through the existing policy. Examine the shared demand as well as individual model traffic: the cited guide does not establish whether a particular 429 is scoped to an account, model or another limit. Moving work to a different model queue is not evidence that the constraint has disappeared.
The decision is therefore not “how many async tasks can Python create?” It is “how many requests should this application admit to its configured CometAPI routes, given observed waiting, latency and failures?” Keep that boundary explicit before increasing concurrency. Async syntax alone proves neither a throughput gain nor a cost reduction.