Skip to content

docs(integration): assign each Lambda case to one validation level - #851

Merged
joeyzhao2018 merged 7 commits into
mainfrom
joey/docs-lambda-handler-test-coverage
Oct 23, 2026
Merged

joeyzhao2018 merged 7 commits into
mainfrom
joey/docs-lambda-handler-test-coverage

Conversation

@joeyzhao2018

@joeyzhao2018 joeyzhao2018 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Consolidates the existing fixture inventory and planned coverage in integration_tests_local/README.md.

  • Assigns each handler, packaging, runtime, or feature case to one lowest sufficient validation level: L1 first, L2 when isolated specs are insufficient, and L3 only for AWS-only behavior.
  • Keeps the separate L1/L2/L3 columns, with exactly one selected column explaining the assertion and placement. The catalog has 21 L1, 25 L2, and 3 L3 assignments; references to shared cases do not request additional tests.
  • Flattens extra dimensions into individual variations and maps all 18 current runnable fixture names into the catalog. Existing tests stay in place.
  • Adds an Implemented column to every row with Yes/Partial/No, evidence, and remaining gaps at the selected level. Tests from dd-trace-js #10775 are explicitly identified as unmerged.

integration_tests/README.md has no changes in this PR.

Motivation

The fixture list and coverage plan should be one guide for deciding where a missing test belongs. Handler return/callback semantics, descriptors, accessors, and timer ownership can be validated directly in unit specs. Actual build artifacts, loaders, dependency instrumentation, and RIC completion behavior need local integration tests. Streaming delivery, AWS extension lifecycle/delivery, and platform-provided trace context require the deployed suite.

Selecting one level per assertion avoids repeating the same validation across L1, L2, and L3. Representative fixtures can cover several relevant variations without testing their Cartesian product.

Testing Guidelines

Documentation only:

  • git diff --check passed.
  • Parsed the README with Prettier's Markdown parser and verified consistent table columns.
  • Checked that all 49 catalog rows have exactly one selected validation level and unique case IDs.
  • Checked all 49 implementation-status comments against existing specs/fixtures and the deployed-suite assertions, and verified their local evidence links.
  • Verified all 18 existing fixture names remain in the catalog, local links/anchors resolve, and the deprecated README matches the base byte for byte.

No unit, Docker/RIE, or AWS tests were run; executable code and fixtures are unchanged.

Additional Notes

Implementation status describes automated test existence at the selected level, not passing results. Temporary manual AWS validations do not count as automated L3 coverage. Follow-up PRs will add only the missing assertions at their selected levels; existing tests are not moved or removed.

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T19:08:57.206358Z 783eeaa PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@joeyzhao2018
joeyzhao2018 requested review from a team as code owners October 22, 2026 19:04

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The guide clearly separates planned coverage from implemented tests and distinguishes native Lambda completion from the library’s Promise adapter. The completed static review identified no actionable documentation defects.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 783eeaa

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 783eeaa0ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread integration_tests/README.md Outdated
@joeyzhao2018 joeyzhao2018 changed the title Joey/docs lambda handler test coverage docs(integration): assign each Lambda case to one validation level Oct 22, 2026
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 60d84c8 | Docs | Give us feedback!

@joeyzhao2018
joeyzhao2018 merged commit 097f711 into main Oct 23, 2026
65 of 67 checks passed
@joeyzhao2018
joeyzhao2018 deleted the joey/docs-lambda-handler-test-coverage branch October 23, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants