A timeout in a concurrent AI API client is not automatically evidence of a slow model response. If the concrete exception is httpx.PoolTimeout, start with connection acquisition in the client—not the read timeout.
The distinction matters when choosing what to investigate. Raising a response-reading budget does not target the budget for waiting to acquire a pooled connection.
Identify the wait that expired
HTTPX’s Timeouts documentation, “Fine tuning the configuration” separates four stages:
| Exception | Wait described by HTTPX | Configuration to inspect |
|---|---|---|
PoolTimeout |
Acquiring a connection from the connection pool | Pool timeout; pool connection limit configured through limits |
ReadTimeout |
Receiving a chunk of data | Read timeout |
ConnectTimeout |
Establishing a socket connection to the host | Connect timeout |
WriteTimeout |
Sending a chunk of data | Write timeout |
For streaming AI responses, keep the read definition precise: it concerns a data chunk, not necessarily a model token or a complete application event. These stage-specific definitions do not establish a total request deadline.
A bounded diagnostic runbook
The following is a suggested investigation procedure, not a claim that one configuration change will fix every PoolTimeout.
- Preserve the actual exception type. Record whether the underlying error is
PoolTimeoutorReadTimeout, rather than grouping both under “provider timeout.” If an SDK exposes only a wrapper error, establish the underlying exception and transport before applying this distinction. - Inspect the client that failed. Record its effective pool and read timeout settings and its
limitsconfiguration, including any relevant request-level timeout override. HTTPX explicitly identifies the maximum allowable pool connections as related to pool acquisition. Do not assume the application’s intended settings are the settings used by that client. - Compare failures with concurrent work. Capture the time of failure and, where observable, the number of in-flight operations using that client. Use this to frame a local pool investigation, not to declare a connection leak or provider saturation. The exception names the expired wait; it does not explain why a connection was unavailable.
- Choose the next change only after that check. For a confirmed
PoolTimeout, investigate the pool-acquisition budget and connection limits first. For a confirmedReadTimeout, investigate the received-chunk wait instead. This source supplies no workload-specific “correct” values for either.
Stop condition: if the underlying transport or exception remains unknown, stop tuning from this table and resolve that uncertainty first. Do not extrapolate these HTTPX definitions to another transport merely because an SDK presents a similar interface.
For escalation, report “pool acquisition timed out,” with the effective configuration and observed concurrency, rather than “the model timed out.” A PoolTimeout alone neither proves a provider response failure nor proves the provider is healthy. Keep the diagnosis at the stage the evidence actually identifies; provider attribution requires additional evidence.