[APMSVLS-485] Send region as a trace stats tag by default

[lucaspimentel]

Please include Jira ticket in title.

Overview

Stacked on #1336. Review that one first; this PR is the last commit only.

With DD_LAMBDA_EXTENSION_COMPUTE_STATS enabled the extension aggregates trace stats itself. The backend then never sees full spans, so it can only use the tags the extension puts on the stats payload, and any tag an org has designated as a primary tag was silently lost on Lambda trace stats. Customers who enabled the setting lost those dimensions on trace.aws.lambda without any indication.

This is also what is behind the two xfailed region primary-tag tests in serverless-e2e-tests#319. Those tests do not measure the tracer, they measure who computed the stats.

The extension now sends region as an additional trace stats tag by default, so it keeps working as a primary tag when the extension computes stats. The value is already parsed from the function ARN and copied into every span, so this adds one value per function and no new cardinality: the aggregation key gains a constant dimension, not a multiplier.

Interaction with DD_TRACE_STATS_ADDITIONAL_TAGS

Setting it replaces the default rather than adding to it:

DD_TRACE_STATS_ADDITIONAL_TAGS	DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED	Result
unset	off (default)	["region"]
unset	on	["region"]
tenant_id	on	["tenant_id"]
region,tenant_id	on	["region", "tenant_id"]
region,tenant_id	off	["region"]

A user who wants region alongside their own keys lists it explicitly. Replacing rather than merging keeps the change clear of the four-key limit on additional stats tags: a one-key default never approaches it, and an explicit list is passed through exactly as written, so users still get all four slots. Merging would have spent one of them on a key the user did not ask for, and since keys over the limit are dropped in alphabetical order rather than by intent, region sorts late enough that it would often have been the one dropped anyway.

The default is deliberately not subject to DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED. That gate exists to keep user-configured keys of unknown cardinality behind a flag; region is neither user-configured nor unbounded.

Note that apply_experimental_features_gate resets to the default rather than clearing. It is the only reset path and runs on every get_config, so clearing there would make the default unreachable in production while still looking correct in tests built from Config::default().

Testing

cargo test --lib, cargo clippy --all-targets -- -D warnings and cargo fmt --check are clean.

Unit tests cover the default reaching the resolved config, the default surviving a disabled experimental gate (the production path, via get_config rather than Config::default()), an explicit setting replacing the default, and region surfacing on exported stats from a span that carries it.

Still to do before this leaves draft:

• Confirm the org-side configuration needed for these tags to be honored is in place for ddserverless.
• Run the statsv2 e2e scenario against a dev layer built from this branch (DD_EXTENSION_ARN), since serverless-e2e-tests pins a published layer.
• Un-xfail the two tests in serverless-e2e-tests and restore the statsv2 rows in FEATURE-PARITY.md. Separate PR, after this ships in a released layer.

🤖

[datadog-datadog-prod-us1-2]

⚠️ Warnings

❌ Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

DataDog/datadog-lambda-extension | cargo fmt — 🔄 Retry may pass, looks flaky

View more details · View in GitLab

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 108eed0 | Docs | View more details | Give us feedback!}

[lucaspimentel]

Oops, my mistake. This is wrong. We don't send region by default, we just have it configured manually in the test org.