Skip to content

Precompute decision maker tag values per sampling mechanism - #12981

Open
dougqh wants to merge 2 commits into
masterfrom
dougqh/ptags-decision-maker-table
Open

dougqh wants to merge 2 commits into
masterfrom
dougqh/ptags-decision-maker-table

Conversation

@dougqh

@dougqh dougqh commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

PTags.installSamplingState built the _dd.p.dm value with TagValue.from("-" + samplingMechanism) on every keep decision. That allocated a String, then ran validation, hashing and a lookup in TagValue's 128-entry cache. All of this happens inside samplingStateLock.

This PR builds the TagValues for mechanisms 0–15 once, at class init, and indexes them by mechanism. Larger mechanisms still go through TagValue.from. The negative/UNKNOWN guard is unchanged.

Motivation

This is the first of several small follow-ups on per-trace sampling overhead in propagation tags. The others are the eager OtelTraceState from #12397 (APMLP-2048), the per-PTags lock object, and a race in the single-entry Knuth sampling-rate cache. Those will come as separate PRs.

The concat itself is older than #12397. #12397 moved it inside the sampling lock.

Benchmark

This PR adds SamplingDecisionBenchmark:

  • span* arms run a bare root span through start → sample → finish.
  • ptags* arms make the decision on fresh PropagationTags with no span around it.

Before = master's PTagsFactory; after = this branch. Both runs used -f 1 -wi 3 -w 2s -i 5 -r 3s -t 8 -prof gc on JDK 17 / M-series laptop. B/op is the primary signal. Throughput comes from a single fork and is directional only.

arm B/op before B/op after Δ B/op ops/µs before ops/µs after
ptagsPriorityKeep 176 152 −24 259 ± 17 319 ± 36
ptagsProbabilityKeep 240 216 −24 185 ± 16 284 ± 47
spanDefaultSamplerKeep 1074 ± 94 1048 ~−26 3.53 ± 0.50 3.51 ± 0.89
spanRuleSamplerKeep 1096 1048 −48 2.87 ± 0.29 2.92 ± 0.12
spanManualKeep 936 840 −96 3.34 ± 1.26 3.31 ± 0.69
  • Each decision-maker update saved 24 B. The span arms save a multiple of that because a trace goes through more than one decision-maker update.
  • The baseline spanDefaultSamplerKeep B/op varied run to run (±94), so treat its Δ as approximate.
  • spanManualKeep also varies between runs: a later run of the same post-change jar measured 888 B/op, not 840. Its Δ could be −48 rather than −96. The ptags* arms gave the same values in every run.
  • The ptags* arms still allocate 150–220 B per trace, for PTags itself, its lock object, SamplingState and the eager OtelTraceState. Those are the targets of the follow-up PRs.

Additional Notes

  • The table holds 16 entries rather than more because each entry also takes a slot in TagValue's shared 128-entry cache.
  • New DecisionMakerTagValueTest covers values inside and beyond the table, and instance reuse.
  • datadog.trace.core.propagation.*, DDSpanContextPropagationTagsTest and CoreSpanBuilderTest pass locally.

🤖 Generated with Claude Code

Every keep decision built "-" + samplingMechanism and then validated, hashed
and looked it up in TagValue's cache. The mechanisms are a small fixed set, so
build their TagValues once and index by mechanism, falling back to
TagValue.from for anything beyond the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh dougqh added tag: no release notes Changes to exclude from release notes type: refactoring comp: context propagation Trace context propagation tag: ai generated Largely based on code generated by an AI or LLM labels Oct 21, 2026
@datadog-prod-us1-4

This comment has been minimized.

Measures the per-trace propagation-tag bookkeeping done on a root sampling
decision, both in a bare start/sample/finish span and on fresh
PropagationTags with no span around it, for the default, rule and manual
keep paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh marked this pull request as ready for review October 21, 2026 13:27
@dougqh
dougqh requested review from a team as code owners October 21, 2026 13:27
@dougqh
dougqh requested review from khanayan123 and mcculls and removed request for a team October 21, 2026 13:27
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T13:29:50.222445Z 4ddb5d9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@dougqh
dougqh requested a review from MilanGarnier October 21, 2026 13:28

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The precomputed decision-maker values preserve existing formatting, the negative-mechanism guard, and fallback behavior for mechanisms outside the table.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 4ddb5d9

@dd-octo-sts

dd-octo-sts Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.93 s 13.90 s [-0.5%; +1.0%] (no difference)
startup:insecure-bank:tracing:Agent 12.90 s 12.94 s [-1.1%; +0.4%] (no difference)
startup:petclinic:appsec:Agent 17.08 s 16.88 s [+0.2%; +2.2%] (maybe worse)
startup:petclinic:iast:Agent 16.31 s 17.06 s [-8.6%; -0.2%] (maybe better)
startup:petclinic:profiling:Agent 16.68 s 16.63 s [-0.5%; +1.1%] (no difference)
startup:petclinic:sca:Agent 17.07 s 16.98 s [-0.3%; +1.3%] (no difference)
startup:petclinic:tracing:Agent 15.68 s 16.27 s [-7.7%; +0.5%] (no difference)

Commit: 4ddb5d9d · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: context propagation Trace context propagation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant