[litianningdatadog]

Overview

Part of SVLS-8713: bottlecap is replacing its own logs aggregator and flusher with the shared datadog-logs-agent crate from serverless-components. Lambda users must not notice the swap, so the plan is: pin today's behavior with tests (this stack), fix two real bugs first so the swap stays a pure refactor (this stack), then swap the aggregator and the flusher (bc-1 and bc-2, which will be opened after the serverless-components PRs merge).

Bug: with Observability Pipelines Worker enabled, every additional logs endpoint was also treated as a worker. Those endpoints are Datadog intakes, so they got requests at the wrong path (no /api/v2/logs), without DD-PROTOCOL, and uncompressed.

Fix: only the primary endpoint follows the OPW setting. Additional endpoints always get /api/v2/logs, DD-PROTOCOL: agent-json, and compression per DD_LOGS_CONFIG_USE_COMPRESSION. Each batch is still compressed only once and shared by every endpoint that sends compressed data; raw batches are freed as soon as nothing needs them.

Fixed before the swap so the swap itself stays a pure refactor (the shared crate already behaves correctly here). User-facing: worth a release note.

Testing

757 tests pass. The contract test that pinned the old behavior now checks the fixed one; a new test covers OPW with compression off. clippy and fmt clean.

Stack

Each PR is nested under the PR it builds on (and targets). Review and merge from the top down.

serverless-components (datadog-logs-agent options; merge first)

• main
  • [SVLS-8713] test(logs-agent): pin current default behavior with characterization tests serverless-components#189 characterization tests (pin today's defaults)
    • [SVLS-8713] build(logs-agent): let users choose the FIPS TLS backend serverless-components#190 default / fips features
      • [SVLS-8713] feat(logs-agent): accept pre-serialized logs and an oversize policy serverless-components#191 pre-serialized logs + oversize policy
        • [SVLS-8713] feat(logs-agent): optional API key resolver and Datadog intake URL override serverless-components#192 API key resolver + intake URL override
          • [SVLS-8713] feat(logs-agent): add a retry-only method and an optional flush policy serverless-components#193 retry() + FlushPolicy
            • [SVLS-8713] perf(logs-agent): compress each log batch only once serverless-components#194 compress each batch once
              • [SVLS-8713] perf(logs-agent): compress log batches before waiting for the API key serverless-components#195 compress before waiting for the API key

datadog-lambda-extension (bottlecap)

• main
  • [SVLS-8713] test(logs): contract tests, fake logs intake and benchmark for the logs pipeline #1448 contract tests, fake intake, benchmark
    • 👉 [SVLS-8713] fix(logs): send additional endpoints the Datadog intake format when OPW is on #1449 fix: OPW applies only to the primary endpoint ← this PR
      • [SVLS-8713] fix(logs): drop a log too large for any payload instead of blocking the queue #1450 fix: drop a log too large for any payload
        • [SVLS-8713] test(logs): pin that a reloaded API key is used on the next flush #1451 pin: reloaded API key is used
          • bc-1: use the shared aggregator (opens after the serverless-components PRs merge)
            • bc-2: use the shared flusher (opens after the serverless-components PRs merge)

🤖 Generated with Claude Code

[datadog-datadog-us1-prod]

 

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

DataDog/datadog-lambda-extension | bottlecap (arm64, fips, alpine) — 🔄 Retry may pass, looks flaky

View more details · View in GitLab

ℹ️ Info

🔄 Datadog auto-retried 2 jobs - 0 passed on retry

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 897463e | Docs | View more details | Give us feedback!}