Skip to content

[WIP] python: select interpreter-compatible coverage defaults - #73

Closed
anmarchenko wants to merge 1 commit into
mainfrom
anmarchenko/python-dependency-compatibility
Closed

anmarchenko wants to merge 1 commit into
mainfrom
anmarchenko/python-dependency-compatibility

Conversation

@anmarchenko

@anmarchenko anmarchenko commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Fixes the action's omitted Python coverage input so it selects 7.10.7 on Python 3.9 and preserves this release's 7.16.1 on Python 3.10–3.14. The tracer default remains ddtrace 4.15.2.

The coverage input has no global default, allowing the installer to distinguish omission from an explicit version. The action sends its modern release pin through DD_DEFAULT_COVERAGE_VERSION_PYTHON and keeps forwarding explicit python-coverage-version overrides unchanged. Normal library-bump helpers update the modern release pin without reinstating a universal coverage default. The summary includes the installed coverage version.

This is a draft coordinated release PR: it points to proposed installer v16 and must wait for the installer artifact to be published and its bytes/checksum verified before release.

Motivation

An uncoached HTTPX onboarding trial using datadog/test-visibility-github-action@v3 on Python 3.9 failed before the test planner started. The action unconditionally supplied coverage==7.16.1. Pip rejected that dependency, then installer v15 misleadingly reported Could not install ddtrace for Python.

Failed Python 3.9 job.

The investigation confirmed:

Dependency Requires-Python Finding
ddtrace 4.15.2 >=3.9,<3.15 Supports Python 3.9–3.14; cp39 wheels exist.
coverage 7.16.1 >=3.10 Incompatible with the action's supported Python 3.9 interpreter.
coverage 7.10.7 >=3.9 Final coverage release supporting Python 3.9.

Coverage 7.11.0 dropped Python 3.9. Pinning that compatibility boundary preserves reproducibility and retains the modern coverage release elsewhere. A moving latest-compatible range would change independently of the action release; globally pinning 7.10.6 would unnecessarily downgrade newer interpreters and merely encode the trial's workaround.

The investigation used action base 1abae93083b69778facd7ee11f89184306c999e6. Downloaded installer v15 matched source in its separate repository and the action's checksum e64a4a45e0b171f4294b9f949f7fc99946192f952a2d463c61fad3cb8b956a58.

Additional Notes

Depends on installer PR: DataDog/test-visibility-install-script#28

Installer source is maintained in DataDog/test-visibility-install-script. The companion change owns the deterministic Python 3.9 pin, separates tracer/coverage failures, uses the selected interpreter's pip, and emits success flags only after installation succeeds. Standalone newer-Python installations retain their existing 7.13.5 coverage default; this action separately supplies 7.16.1.

Explicit input behavior:

  • Python 3.9 + python-coverage-version: 7.10.6 installs exactly 7.10.6.
  • Python 3.9 + python-coverage-version: 7.16.1 fails with Could not install coverage==7.16.1 for Python 3.9.16, retains pip's diagnostics, and does not fall back or advertise successful instrumentation.

Release sequence:

  1. Merge/validate the installer PR and publish v16 through that repository's existing manual GitLab deployment pipeline.
  2. Download the published artifact and verify it against reviewed source. The proposed checksum here is be1de997fab7862640146b6836c96cbf13f4e9fa577cb501e80c37c5a5c60768; update it if source bytes change.
  3. Run this action's hosted Python matrix, then release through the documented action release process.

The v16 URL returned HTTP 404 on 7 October 2026. Until deployment, the hosted action matrix is expected to fail at artifact download/checksum verification. Do not merge/release this action against an unavailable artifact, and do not overwrite v15, whose bytes remain required by existing immutable action releases.

Possible Drawbacks / Trade-offs

Python 3.9 stays on coverage's final compatible release. Future dependency bumps can change interpreter support, so the new Python matrix must pass before release. Explicit incompatible versions fail instead of silently changing a user's request.

The action input is now optional without a YAML default; maintainers must bump DD_DEFAULT_COVERAGE_VERSION_PYTHON instead. Both bump helpers and a regression guard are updated accordingly. Existing user version inputs remain authoritative.

Hosted Linux action behavior and Datadog event delivery remain unverified. Windows was excluded as requested. This patch does not address the trial's baseline HTTPX Twine metadata/trio warning failures or the separate hidden .coverage.* upload issue.

Describe how to test/QA your changes

Completed locally on macOS arm64, using the companion installer's final bytes with this action's environment settings:

Python ddtrace Coverage Instrumented pytest fixture Statement/branch coverage
3.9.16 4.15.2 7.10.7 2 passed 100%
3.10.16 4.15.2 7.16.1 2 passed 100%
3.11.17 4.15.2 7.16.1 2 passed 100%
3.12.9 4.15.2 7.16.1 2 passed 100%
3.13.15 4.15.2 7.16.1 2 passed 100%
3.14.8 4.15.2 7.16.1 2 passed 100%

Each fresh environment asserts that the active Datadog TestOptPlugin is registered. Tracing used a dummy API key and a closed localhost endpoint. These are smoke-fixture coverage results, not a rerun of HTTPX's full application suite or proof of backend delivery.

Additional validation:

  • Python 3.9 explicit 7.10.6 remained exact, with active instrumentation, 2 tests passing, and 100% statement/branch coverage.
  • Python 3.9 explicit 7.16.1 failed with the correct dependency diagnostic and no instrumentation environment output.
  • Replayed the actual composite download/run step on Python 3.9 and 3.14, with download stubbed to the checksum-verified local installer. Both success cases ran instrumented pytest; the incompatible Python 3.9 override left GITHUB_ENV empty. This was local wiring validation, not a hosted action run.
  • The new bump guard passed two successive coverage bumps and a tracer bump in disposable copies, preserving the empty coverage input default and Python 3.9 pin.
  • The companion installer's 7 deterministic tests and 16 existing Go scenarios passed locally. After opening the installer PR, its hosted Ubuntu Python matrix passed on all six versions, 3.9–3.14, and its Go matrices passed on Ubuntu and macOS. This action's hosted defaults bump guard also passed; its installation checks remain blocked by the unavailable v16 artifact.
  • Bash syntax, ShellCheck at error severity, and YAML parsing passed. Both proposal patches applied cleanly to their recorded bases.

Run the local bump guard:

tests/test-python-defaults.sh

After publishing v16, the new hosted action matrix exercises defaults on Python 3.9–3.14, the compatible 7.10.6 override, and the incompatible 7.16.1 override. It verifies selected versions, active pytest instrumentation, fixture coverage, and absence of success flags on expected failure.

@anmarchenko anmarchenko added the semver-patch Requests a patch test-visibility-github-action release after merge label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-patch Requests a patch test-visibility-github-action release after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant