CometAPI retry classifier: make error.code a stop signal
A 500 is not automatically permission to retry. In CometAPI’s documented error envelope, the HTTP status sits alongside error.code, error.message, and request context. That means the retry gate should classify the body before it spends another attempt.
The stop branch
Stop and escalate for correction when the response is a 401, a path-related 404, or a 500 carrying invalid_request or invalid_request_error. A 401 points to authentication configuration; a 404 usually requires checking the CometAPI base URL and route. A malformed request remains malformed when sent to another model. Repeating it or switching models only obscures the defect.
For a 500 with invalid_request, record the request ID and the rejected parameter, fix the payload, and only then re-enter the classifier. Do not retry the unchanged body.
The bounded retry branch
Treat 429 and temporary 5xx responses such as 503, 504, and 524 as retry candidates. Apply exponential backoff with jitter, reduce burst concurrency when rate limiting is involved, and count retries and fallback attempts against one total latency budget. After a small, explicit retry cap, a fallback route is reasonable only if it supports the same input, output, and required capability contract.
The fallback decision is separate from the retry decision. Before changing the model, verify structured-output, tool, multimodal, or provider-specific requirements rather than assuming a model ID swap is compatible. If a prior attempt may have triggered a side effect or returned a partial stream, resume from a checkpoint or fail in a controlled way instead of blindly repeating the whole request.
Signals worth logging
For each decision, log status, error code, request ID, model, attempt, delay, and fallback reason without secrets. Alert on repeated stop-branch failures and a rising fallback rate. Those signals identify configuration or route problems; they are not evidence of provider availability.
The operational rule is simple: status chooses the candidate branch, but error.code can veto retry. Let authentication, path, and malformed-request errors fail loudly; back off boundedly for rate limits and temporary server failures; and make fallback conditional on a tested, equivalent contract.