When pointing a LiteLLM Proxy deployment at CometAPI, keep the inference address and the database connection in separate change sets. The useful baseline is simple: change the model route while holding storage configuration constant, then test each dependency separately.
The LiteLLM Getting Started example puts these fields in different places:
| Field | Where the example puts it | Decision for a CometAPI integration |
|---|---|---|
api_base |
model_list[].litellm_params |
Configure the model’s upstream inference address here, after confirming the serving base and adapter requirements. |
database_url |
general_settings, with a postgres:// value |
Keep this as a separately managed database connection, not a place to enter a CometAPI HTTP endpoint. |
That example uses Azure, not CometAPI. The CometAPI connection comes from its official LiteLLM integration guide, which describes the api_base/api_key override for OpenAI-compatible inference and names /v1/chat/completions. Together, the two sources support a configuration boundary—not a claim that every proxy feature works unchanged through CometAPI.
Don’t turn the guide into an untested deployment config
There is a reason not to paste the CometAPI article’s basic-call snippet directly: it labels a console URL as api_base and includes a malformed os.environ assignment. I would treat its integration description as a starting point, not proof of the current serving URL, model prefix, or installed LiteLLM version’s support.
Before changing the proxy route, confirm those three details against the current integration documentation. If the serving base is still ambiguous, stop there rather than guessing a replacement URL. None of that uncertainty is a reason to change database_url.
A reproducible check for the route change
My suggested acceptance check is deliberately narrower than a full application evaluation:
- Record the LiteLLM version, existing model-route configuration, and a minimal non-sensitive request as the baseline. Keep secrets out of the comparison record.
- Configure the confirmed CometAPI serving base, model identifier, and credential reference in the inference configuration. Leave the database connection unchanged.
- Send the same minimal request through the proxy and check that the intended route accepts it and returns the response shape the application expects. Check the database connection independently; a successful model response is not evidence that storage works, or vice versa.
- If the inference check fails, investigate the serving address, authentication, model identifier, and adapter compatibility before making unrelated database changes.
These are proposed checks, not reported test results. The cited proxy example also does not tell us which records are persisted, whether a database is required for a particular deployment, or what retention guarantees exist. In a retrieval application, do not infer from the presence of database_url that it is the corpus or vector store. That needs its own documented configuration and validation.
The practical win is an interpretable change: “the CometAPI inference route passed this request” stays a separate statement from “the proxy’s database connection passed its check.” Neither statement, on its own, establishes retrieval quality.