Why Replayability Should Decide Your API Retry Policy

General
1

Why Replayability Should Decide Your API Retry Policy

Automatic retries are safest when the client can send the same request body again. It is easy to make “transient error” the only condition in a retry policy and forget that the body may be one-shot.

The OpenAI Python SDK v3.19.0 release notes document a narrow client change: “retry only replayable request content.” The note does not promise that every failed request will be recovered, and it does not spell out the behavior of every endpoint or payload type. It does establish a useful integration boundary: retry eligibility depends on whether the request content can be replayed.

Make replayability an input to the policy

Before an API call enters an automatic retry loop, classify the body:

  • Replayable: the application can produce the same bytes or equivalent request content again without consuming or changing its source. This is the case where SDK-managed retries are a reasonable default, subject to the SDK’s normal limits and the endpoint’s semantics.
  • Not known to be replayable: the body comes from a stream, a one-shot iterator, or another source that may already have been consumed. Do not assume a transport-level retry can reconstruct it.

For the second category, move the decision into application code. Buffer the content first if the memory and privacy trade-offs are acceptable, or build a recovery path that can recreate the request from an authoritative source. If neither is possible, surface the failure rather than silently replaying an incomplete or different body.

That is a policy choice, not a claim that a particular payload class is always handled one way by every SDK version. The release note names replayable request content as the condition; your integration still needs to verify how its body producer behaves.

Separate transport acceptance from effect

A retry response is not proof that the intended application effect happened exactly once. Keep the retry decision separate from effect verification:

  1. Decide whether the body is replayable before issuing the request.
  2. Let the client retry only within that safe boundary.
  3. For non-replayable content, preserve enough context to decide whether to recreate, resume, or stop.
  4. Use the API’s documented response and application-level identifiers to verify the result where the operation requires it.

This matters when code treats a timeout or connection reset as evidence that nothing happened. A failed read does not, by itself, prove the server did not accept the request. Replayability answers whether sending the body again is safe; it does not answer whether the first attempt took effect.

A small review checklist

When reviewing an SDK upgrade or retry wrapper, ask:

  • Can the body producer generate the same content after the first attempt?
  • Is that property explicit in the integration, or merely assumed from the type name?
  • What should happen when the body cannot be recreated: buffer, resume, reconcile, or fail?
  • Which response or identifier will you use to verify the intended effect after an uncertain outcome?

The v3.19.0 note is a reminder to make that boundary explicit. Treat replayability as a prerequisite for automatic retries, and keep recovery and effect verification as separate decisions.