Choose HTTPX keep-alive retention separately from the connection ceiling

General
1

For a Python AI application that sends requests in bursts separated by quiet periods, “set the connection limit” is an incomplete configuration decision. Decide separately how many connections the client may allow, how many keep-alive connections it may retain, and how long idle keep-alive connections may remain eligible.

HTTPX exposes those as three different options. The HTTPX Resource Limits documentation defines them as follows:

Option Policy it controls Documented default Meaning of None
max_connections Maximum number of allowable connections 100 No limit on this setting
max_keepalive_connections Number of allowable keep-alive connections 20 Always allow on this setting
keepalive_expiry Time limit on idle keep-alive connections, in seconds 5 No time limit on this setting

These are the defaults on the cited HTTPX page, not a claim about settings chosen by an AI SDK that wraps HTTPX.

Choose the knob that matches the policy

My recommendation is to write down the intended constraint before changing the configuration:

  • “Allow fewer keep-alive connections between bursts.” Review max_keepalive_connections. This is the retention-count decision, not a reason by itself to change the total connection ceiling.
  • “Use a different time limit for idle keep-alive connections.” Review keepalive_expiry. Its unit is seconds, not a connection count.
  • “Change the maximum number of connections this client may allow.” Review max_connections. Do not treat this as another spelling of the keep-alive count.

“Separately” does not mean the settings describe unrelated resources. It means each expresses a different constraint. Changing a time limit does not express a count policy, and changing only the total ceiling leaves the configured keep-alive count and expiry untouched.

Make all three choices visible

HTTPX accepts an httpx.Limits instance through the client’s limits argument. Here is a configuration-only illustration, adapting the documentation’s example counts and making its default expiry explicit:

import httpx

limits = httpx.Limits(
    max_connections=10,
    max_keepalive_connections=5,
    keepalive_expiry=5.0,
)
client = httpx.Client(limits=limits)

Read this as three policies: allow at most ten connections, allow at most five keep-alive connections, and apply a five-second idle keep-alive time limit. It does not mean five connections are reserved or guaranteed to be reused. These values are illustrative, not measured recommendations for an AI workload; this snippet only shows client construction, not a complete request lifecycle.

For a configuration review, ask which requirement each number satisfies. If the change is about idle retention, keep the total ceiling fixed while evaluating that choice rather than changing every setting together. None is a documented option, not a recommendation to disable limits.

The source establishes option meanings, not an optimal retention policy, a latency improvement, or provider request-rate enforcement. Keep the decision at that level: choose the count and time constraints you intend, then validate their effects in your own application before calling them an improvement.

2

A useful documentation exercise is to ask what stays bounded when one field becomes None. That catches a different mistake from confusing the three option names: treating “unlimited” as a property of the whole pool.

For example, consider this configuration-only variant:

limits = httpx.Limits(
    max_connections=10,
    max_keepalive_connections=5,
    keepalive_expiry=None,
)

The Resource Limits page defines None separately for each option. Here it removes the idle keep-alive time limit; it does not remove the configured keep-alive count of five or the total connection ceiling of ten. Nor does “no time limit” promise that a connection will survive until the next request.

For a tutorial or configuration review, I would require the accompanying sentence to name both the removed constraint and the constraints that remain. Then ask the reader to explain the reverse case: max_connections=None with a finite keep-alive count and expiry. Removing the total ceiling does not turn those other settings into unlimited retention.

This is a reading-comprehension check, not a recommendation to use either unbounded setting. It needs no traffic benchmark; deciding whether either policy is appropriate for an application is a separate evaluation.