Skip to content

Keep global tags apart from span defaults in the config snapshot (OTLP trace/resource-level split) - #12937

Open
dougqh wants to merge 7 commits into
masterfrom
dougqh/split-global-span-tags
Open

dougqh wants to merge 7 commits into
masterfrom
dougqh/split-global-span-tags

Conversation

@dougqh

@dougqh dougqh commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Keeps the global tags (DD_TAGS) apart from the span defaults (DD_TRACE_SPAN_TAGS) in the tracer's config snapshot. Remote config tracing_tags behaves as before.

  • Dynamic config: tracingTags now holds only the global tags, seeded from Config.getGlobalTags(). The test-runner path, rebuildTraceConfig, already did this.

  • Span defaults: DD_TRACE_SPAN_TAGS (new Config.getSpanTags()), or the tags given to the builder's defaultSpanTags.

  • ConfigSnapshot: keeps two frozen maps alongside mergedTracerTags:

    • globalTags: the global tags as given, or their RC override. The override keeps the configured env and version;
    • spanDefaultTags: the span defaults plus the tracer-defined tags (env, _dd.dsm.enabled, …). While an RC tracing_tags override is active, only the tracer-defined tags.

    mergedTracerTags is built from both, with span defaults winning as before. Read-through and span semantics are unchanged.

Motivation

#12813 (OTLP): to put DD_TAGS on the OTLP Resource without repeating them on every span, the exporter needs the global tags as their own set. Today it has to recover them by comparing values against the static Config. With this change, core can build the Resource from globalTags and write spanDefaultTags on spans, taking both from the trace's snapshot (TraceCollector.getTraceConfig()). The Resource then also follows RC tag changes, and keeps env and version through them. The OTLP side is a follow-up and isn't included here.

Behavior changes (release notes)

Additional Notes

  • Stacked on Add test for remote config tracing tags overriding DD_TAGS (OTLP trace/resource-level split) #12936. Review the commits on top of it.
  • Cold path only: the maps are rebuilt only when a config snapshot's tracing tags change. Per-span work is unchanged.
  • env and version in globalTags: Config.getGlobalTags() includes them when DD_ENV / DD_VERSION are set, and an RC override keeps them, so a Resource built from globalTags doesn't lose them while an override is active.
  • No production reader of globalTags / spanDefaultTags yet. The consumer is the Stop repeating global tags as OTLP span attributes #12813 rebuild.
  • Tests:
    • remoteConfigTracingTagsOverrideGlobalTags has a DD_TRACE_SPAN_TAGS row, expecting the override to replace span tags too;
    • new configSnapshotSeparatesGlobalTagsFromSpanDefaults, covering the override and its removal;
    • both use injectSysConfig, so each table row can set its own config.

🤖 Generated with Claude Code

dougqh and others added 3 commits October 19, 2026 13:22
Remote config tracing_tags replaces the global tags (DD_TAGS), and
removing the override restores them. This pins the intent stated in
system-tests' TestDynamicConfigV2.test_tracing_client_tracing_tags,
which only asserts that the RC tags are present.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remote config tracing_tags now replaces only the global tags (DD_TAGS).
Previously the dynamic config held DD_TAGS and DD_SPAN_TAGS merged, so an
RC tag override also dropped DD_SPAN_TAGS until it was removed.

ConfigSnapshot now keeps the global tags and the span defaults as separate
frozen TagMaps, and builds mergedTracerTags from them (span defaults win,
as before). Span defaults are DD_SPAN_TAGS, or the tags given to the
builder's defaultSpanTags, and are no longer lost on the first RC tag
change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh dougqh added type: bug fix Bug fix comp: core Tracer core comp: remote config Configuration at Runtime tag: ai generated Largely based on code generated by an AI or LLM labels Oct 19, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 50.00%
• Overall Coverage: 59.47% (-0.39%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1e46ce6 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.73 s 14.69 s [-0.5%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 13.51 s 13.67 s [-1.8%; -0.5%] (maybe better)
startup:petclinic:appsec:Agent 17.15 s 17.08 s [-0.5%; +1.4%] (no difference)
startup:petclinic:iast:Agent 17.01 s 17.11 s [-1.3%; +0.2%] (no difference)
startup:petclinic:profiling:Agent 16.66 s 16.98 s [-3.0%; -0.8%] (maybe better)
startup:petclinic:sca:Agent 17.09 s 16.92 s [+0.2%; +1.8%] (maybe worse)
startup:petclinic:tracing:Agent 16.27 s 16.10 s [-0.0%; +2.2%] (no difference)

Commit: 1e46ce6f · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh
dougqh marked this pull request as ready for review October 19, 2026 23:55
@dougqh
dougqh requested a review from a team as a code owner October 19, 2026 23:55
@dougqh
dougqh requested review from BridgeAR and removed request for a team October 19, 2026 23:55
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T23:57:35.775754Z 3d144d9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@dougqh
dougqh requested review from mcculls and mhlidd October 19, 2026 23:56

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Remote-config tracing-tag replacements preserve local span defaults, including their precedence over global tags. The completed static review identified no reportable regression.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 3d144d9

@dougqh dougqh changed the title Keep DD_SPAN_TAGS when remote config overrides tracing tags Keep DD_SPAN_TAGS when remote config overrides tracing tags (OTLP trace/resource-level split) Oct 20, 2026
Comment thread dd-trace-core/src/main/java/datadog/trace/core/CoreTracer.java
Comment thread dd-trace-core/src/main/java/datadog/trace/core/CoreTracer.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/CoreTracer.java Outdated
…nv and version

Keeps the split of global tags and span defaults in the config snapshot,
but restores the existing remote config behavior: while a tracing_tags
override is active, it replaces DD_TRACE_SPAN_TAGS too. Whether span
defaults should survive an override is left for a separate change.

The override's global tags keep the configured env and version, so a
consumer of the global tags (the OTLP resource) still sees them.

Also names the span tags setting DD_TRACE_SPAN_TAGS, and switches the
remote config tests to injectSysConfig.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh dougqh changed the title Keep DD_SPAN_TAGS when remote config overrides tracing tags (OTLP trace/resource-level split) Keep global tags apart from span defaults in the config snapshot (OTLP trace/resource-level split) Oct 21, 2026
Base automatically changed from dougqh/rc-tracing-tags-tests to master October 21, 2026 13:22

@mcculls mcculls left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks for the cleanup

I noted a small discrepancy for an edge-case where the RC tracing_tags override is exactly the same as DD_TAGS but that's not a blocker to merging this improvement.

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

An RC tracing_tags override equal to the configured global tags fails to replace span defaults. This includes an empty override intended to clear DD_TRACE_SPAN_TAGS when DD_TAGS is unset.

Open Bits AI session

🤖 Bits Code Review · Commit 734aa30

Comment thread dd-trace-core/src/main/java/datadog/trace/core/CoreTracer.java Outdated
dougqh and others added 2 commits October 21, 2026 13:43
…-span-tags

# Conflicts:
#	dd-trace-core/src/test/java/datadog/trace/core/CoreTracerTest.java
Deciding whether an override was active by comparing the tags to DD_TAGS
missed an override equal to them, including an empty override when no
global tags are set. Mark tags set from remote config in DynamicConfig
and use that, also when deciding whether to reuse a snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core comp: remote config Configuration at Runtime tag: ai generated Largely based on code generated by an AI or LLM type: bug fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants