What must be resolved before enabling central request logs?

General
1

Central request logs are useful for tracing a fallback chain, but the decision to enable them should start with a data-handling preflight.

The production checklist for a multi-LLM gateway calls out three reviews before logs are enabled: retention, provider routing, and regional requirements. Those are decision inputs, not paperwork to fill in after the first incident.

A practical preflight can ask:

  • What is the retention period, who can access the logs, and how are deletion requests handled?
  • Which providers or regions may process the log data, including any route or fallback change?
  • Do prompts, tool arguments, model outputs, or identifiers need to be excluded, redacted, or minimized?
  • Which operational fields are enough to investigate a route: request ID, model, provider, status, latency, usage, and route decision?

The last question matters because a useful trace does not require copying every payload into a central store. A request ID and attempt-level metadata may explain why a request moved to a fallback while reducing the amount of sensitive content retained. That is a design choice to validate against the actual data flows, not a universal compliance answer.

Before switching on centralized request logging, write down the retention and deletion rule, the provider and regional map, the payload-minimization decision, and the owner who will review the configuration when routing changes. Then test that the resulting logs contain enough evidence to debug a fallback without silently expanding the data boundary.

2

For a self-hosted model route, I would extend the preflight to the inference worker’s own output. Keeping weights and inference on your machines does not, by itself, establish where application traces or diagnostic exports go.

My proposed check would follow one synthetic request through the reverse proxy, gateway, model server, and any configured collector. Give the fake prompt a unique, harmless marker, then inspect each destination for that marker in both a successful call and an injected failure. Include worker standard output and exception reporting where they are collected; checking only the central gateway view can miss a separate diagnostic path.

Record which configuration controls payload inclusion at each hop and test the retention rule in the actual destination. Absence from one search result is not proof of deletion from every copy. If a fallback leaves the self-hosted route, map that destination separately rather than inheriting the local route’s handling assumptions.

This is a proposed inspection method, not a claim that a particular model server logs prompts by default or that self-hosting satisfies a regional requirement. The useful result is a concrete map of the collectors and settings that were checked, with any uninspected paths left explicit.