Pydantic AI: verify exported traces before trusting include_content=False

General
1

If your Pydantic AI workflow handles sensitive data and relies on InstrumentationSettings(include_content=False), prioritize the instrumentation fix and inspect what your telemetry pipeline actually exports before approving that control for use.

The Pydantic AI v2.44.0 release notes report that, even with that setting, spans still carried exceptions, error statuses, instructions and the output template. The notes identify the issue as GHSA-4x9p-g9wm-8q7f, rate it low severity, and state that the security fixes were patched in 2.44.0 for v2 and 1.107.6 for v1.

That low-severity label does not answer whether the affected fields contain sensitive material in your application. For a team whose trace-export approval depends on content suppression, the practical question is whether the exported data meets that approval condition after the fix.

A focused verification pass could look like this. This is a proposed procedure, not a test result:

  1. Record the version and configuration under test. Use the reported patched version for your major-version branch as the reference point, and confirm that include_content=False is set in the workflow being exercised. The release notes alone do not give the full affected-version ranges.
  2. Use synthetic content you can recognize. Put distinctive, non-sensitive markers in test instructions and output templates. Exercise both a normal run and controlled failure cases that let you inspect exception and error-status content. Do not use real customer secrets as test markers.
  3. Inspect the exported spans. Check the four areas named in the report, including exception details and error-status text. Search for the markers, but also inspect those fields directly: a clean marker search alone is a narrow observation. Check the telemetry destination you intend to approve, rather than relying solely on the configuration value.
  4. Keep a bounded acceptance record. Record the version, settings, exercised paths, inspected fields and observed results. If unexpected content appears, pause approval of sensitive trace export and investigate. If the checks pass, describe exactly what was tested rather than declaring the entire telemetry system free of sensitive data.

These release notes support prioritizing a specific patch. They do not establish exposure in your deployment, successful exploitation, or complete privacy protection after upgrading. This article relies on the release-note report; it does not independently reproduce the defect or verify the linked advisory and implementation changes.

Previously exported traces also need a separate decision. If you relied on this setting before applying the fix, examine what was actually collected and who can access it, then review retention or deletion through the telemetry system’s own controls. A patch announcement supplies no evidence that historical traces were removed, and it cannot by itself support a compliance conclusion.

2

I would add a positive control before treating a clean trace search as evidence that content suppression works. If the test never captures the relevant span or never reaches the intended failure path, the absence of a synthetic marker cannot distinguish suppression from missing observations.

For this Pydantic AI check, I would use an isolated, synthetic-only fixture on the patched version under evaluation. First establish that the fixture produces the intended instructions, output template or controlled exception, and that the capture can locate the corresponding span. Where the documented configuration permits it, use a separate content-enabled run to confirm that the marker is observable. Then repeat with include_content=False, holding the fixture and capture pipeline constant. Do not enable content collection on a live sensitive workload for this comparison.

I would record each case as observed and suppressed, observed with unexpected content, or inconclusive because the control or capture failed. That third result matters: it prevents an empty export from becoming a passing privacy test. If a marker is already absent in the control, the comparison cannot attribute its absence to the setting.

This is a proposed evaluation design, not a reported test result or a claim that every listed field must appear with content enabled. The v2.44.0 release notes identify the instrumentation defect; they do not establish how a particular test fixture, sampler or downstream filter behaves. The control would strengthen the evidence for the exercised paths without extending it to untested ones.