[litianningdatadog]

Overview

Part of SVLS-8713: bottlecap is replacing its own logs aggregator and flusher with the shared datadog-logs-agent crate from serverless-components. Lambda users must not notice the swap, so the plan is: pin today's behavior with tests (this stack), fix two real bugs first so the swap stays a pure refactor (this stack), then swap the aggregator and the flusher (bc-1 and bc-2, which will be opened after the serverless-components PRs merge).

Bug: a single log larger than the 5 MiB payload limit could never be sent and stayed at the front of the queue forever, so every log after it was never sent either.

Fix: such a log is dropped with one warning and the batch keeps filling with the logs behind it. Dropped only when it can't fit even alone (len + 2 > 5 MiB, the same threshold as the shared crate); logs between 1 MiB and that limit are still sent with a warning, as before.

User-facing: worth a release note.

Testing

760 tests pass. The contract test that pinned the stuck queue now checks that the logs around the huge one are sent; new unit tests cover the size boundary. clippy and fmt clean.

Stack

Each PR is nested under the PR it builds on (and targets). Review and merge from the top down.

serverless-components (datadog-logs-agent options; merge first)

• main
  • [SVLS-8713] test(logs-agent): pin current default behavior with characterization tests serverless-components#189 characterization tests (pin today's defaults)
    • [SVLS-8713] build(logs-agent): let users choose the FIPS TLS backend serverless-components#190 default / fips features
      • [SVLS-8713] feat(logs-agent): accept pre-serialized logs and an oversize policy serverless-components#191 pre-serialized logs + oversize policy
        • [SVLS-8713] feat(logs-agent): optional API key resolver and Datadog intake URL override serverless-components#192 API key resolver + intake URL override
          • [SVLS-8713] feat(logs-agent): add a retry-only method and an optional flush policy serverless-components#193 retry() + FlushPolicy
            • [SVLS-8713] perf(logs-agent): compress each log batch only once serverless-components#194 compress each batch once
              • [SVLS-8713] perf(logs-agent): compress log batches before waiting for the API key serverless-components#195 compress before waiting for the API key

datadog-lambda-extension (bottlecap)

• main
  • [SVLS-8713] test(logs): contract tests, fake logs intake and benchmark for the logs pipeline #1448 contract tests, fake intake, benchmark
    • [SVLS-8713] fix(logs): send additional endpoints the Datadog intake format when OPW is on #1449 fix: OPW applies only to the primary endpoint
      • 👉 [SVLS-8713] fix(logs): drop a log too large for any payload instead of blocking the queue #1450 fix: drop a log too large for any payload ← this PR
        • [SVLS-8713] test(logs): pin that a reloaded API key is used on the next flush #1451 pin: reloaded API key is used
          • bc-1: use the shared aggregator (opens after the serverless-components PRs merge)
            • bc-2: use the shared flusher (opens after the serverless-components PRs merge)

🤖 Generated with Claude Code

[datadog-datadog-us1-prod]

🔄 Datadog auto-retried 2 jobs - 2 passed on retry

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3f01719 | Docs | View more details | Give us feedback!}