[ICPRO-41] Add SDK_CONFIGURATION RC parametric coverage - #7640
Conversation
Validates that a tracer advertising the SDK_CONFIGURATION RC capability consumes a migrated setting delivered via the generic sdk_config block and produces behavior identical to the equivalent lib_config delivery, starting with DD_TRACE_ENABLED. No tracer implements the capability yet, so it's gated missing_feature across all languages until one ships. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Uses the tracer's reported/telemetry config value as a proxy for "the setting was applied" since system-tests has no signal for real profiling activity (no mock profiling intake, no RC-driven E2E profiling coverage). Reuses the existing SDK_CONFIGURATION test class, which is already gated missing_feature in every language manifest. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
Drop the dd-go implementation-detail reference, stop framing this as a PoC, describe sdk_config in terms of behavior compatibility rather than being a lib_config mirror, and rename the profiling test since it has no lib_config equivalent to compare against. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
test_library.config() is missing several fields and is permanently empty for nodejs by design, so it can't serve as a cross-language echo mechanism. Switch to the tracer's real telemetry configuration (reused from test_telemetry.py's name mapping) for both the profiling test and a new parametrized test covering the remaining migrated lib_config fields (sample rate, logs injection, header tags, tags, data streams, dynamic instrumentation) with non-default values. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Taking the highest-seq_id entry passed only because RC happens to be the highest-precedence source, not because we verified the value actually came from RC. Filter on origin="remote_config" explicitly so a tracer that silently ignores sdk_config can't pass by coincidence. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e per-version support from capabilities.yml Adds telemetry_name_mapping entries for tracing_sampling_rules, code_origin_enabled, exception_replay_enabled, and live_debugging_enabled (previously missing, which had silently excluded them from _SDK_CONFIG_FIELDS despite each having a real capability bit). Collapses the per-field tests into a single test that iterates the full backend-level field list and only exercises fields the current tracer version's legacy APM_TRACING_* bits cover, read live via get_expected_capabilities_for_version -- capabilities.yml is version-range-based and purely additive, so it keeps answering "did this exact version support the legacy bit" even after a later version migrates to SDK_CONFIGURATION. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…play/live_debugging TestDynamicConfigSdkConfiguration is missing_feature-gated off for every language today, so nothing exercises these fields yet. Adding their telemetry_name_mapping entries speculatively risks a wrong key silently passing/failing rather than a missing one loudly erroring -- better to add each field, with its mapping entry, when the language that unblocks this class actually needs it covered. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
When _assert_telemetry_config_applied can't find the expected telemetry config under any known key, hint that the tracer may report it under a name not yet present in test_telemetry.py's telemetry_name_mapping -- this is the signal that should surface it when a language starts reporting a currently-deferred capability bit (see the comment above _SDK_CONFIG_FIELDS), rather than a separate capability-tracking check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…l of system-tests The E2E tests/test_profiling.py suite does validate real profiler payloads via interfaces.agent/interfaces.library; only parametric lacks a mock profiling intake to capture such payloads from. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…plied docstring _SDK_CONFIG_FIELDS is defined at module level above this class, not below this docstring -- name it directly instead of a vague, wrong spatial reference. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Drop the paragraph explaining capabilities.yml's additive/version-range design -- that belongs with get_expected_capabilities_for_version, not repeated at every call site. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…skip message - _SDK_CONFIG_FIELDS is restored to the complete backend-level truth (all 10 LibConfig settings with an APM_TRACING_* capability bit). telemetry_name_mapping stays incomplete for tracing_sampling_rules/code_origin_enabled/ exception_replay_enabled/live_debugging_enabled -- a language that actually supports one of these will surface the gap via _assert_telemetry_config_applied's failure message rather than being silently excluded here. - Drop comments that restated the above or pointed at stale/confusing references. - Reword the "nothing applicable" skip message to be explicit that the test ran nothing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Explain in the class docstring how the single deep DD_TRACE_ENABLED behavioral check combines with the per-field telemetry-only checks to give regression confidence without a custom assertion per setting.
get_telemetry_config_by_origin's return type is a union of dict/str/int/bool/None, so indexing the result with ["value"] doesn't type-check. Use return_value_only=True to get the scalar directly instead.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f91c9e4f6f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…config test class - Stop clearing telemetry/RC state before reading it in the sdk_config tests; clear only after each field's assertion so per-iteration isolation doesn't destroy data the assertion still needs. - Replace the live capabilities.yml lookup in test_sdk_config_field_is_applied with a frozen snapshot, since capabilities.yml is expected to drift as tracers migrate settings to sdk_config and drop their old APM_TRACING_* bits. - Rename TestDynamicConfigSdkConfiguration to Test_DynamicConfigSdkConfiguration per naming convention, and re-sort the affected manifest entries accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
code_origin_enabled, live_debugging_enabled, and tracing_sampling_rules had no mapping entry, so Test_DynamicConfigSdkConfiguration always failed to find the tracer's telemetry config regardless of whether it was actually applied correctly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Avoids a cross-test-file import between test_dynamic_configuration.py and test_telemetry.py, per PR review (CI optimization requirement). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Motivation
Tracers are migrating away from reporting one
APM_TRACING_*Remote Config capability bit per setting, toward a singleSDK_CONFIGURATIONbit that covers a generic, env-var-keyedsdk_configpayload (a sibling of the legacylib_configpayload). As each tracer makes that switch, we want confidence that settings already covered by the legacy per-field bits keep behaving the same way once delivered viasdk_configinstead — i.e. that the migration doesn't quietly change behavior.RFC: https://docs.google.com/document/d/13aKzR8U97-yzaJZDs8rnNVGn4C1NCmfPhAoDkBtH9NM/edit?tab=t.0#heading=h.f9tdtrzge1bf
This PR adds parametric coverage for that: a tracer that declares
SDK_CONFIGURATIONshould applysdk_configthe same way it used to apply the equivalentlib_configsetting.What changed?
TestDynamicConfigSdkConfigurationintests/parametric/test_dynamic_configuration.pyis a new test class covering thesdk_configdelivery path.test_capability_sdk_configurationandtest_sdk_config_tracing_enabled_matches_lib_configconfirmDD_TRACE_ENABLEDdelivered viasdk_configdisables tracing the same way the legacylib_config/env-var path does.test_sdk_config_profiling_enabledcoversDD_PROFILING_ENABLED, which has nolib_configequivalent to compare against (it only exists insdk_config), so it just checks the tracer reports the setting as applied.test_sdk_config_field_is_appliedis a single parametrized test that loops over everylib_configsetting with a correspondingAPM_TRACING_*capability bit, and, for each one this tracer version actually declares (checked live againstcapabilities.yml), delivers it viasdk_configand asserts it was applied — using non-default test values so a tracer that silently no-ops onsdk_configcan't pass by accident. Not every one of those settings has atest_telemetry.pyname-mapping entry yet; where one's missing, the assertion fails with a message pointing at the gap, so whichever tracer next needs that setting covered will discover it there rather than the coverage being silently skipped.Since no tracer implements
SDK_CONFIGURATIONtoday, the whole class is gated asmissing_featurein every language's manifest.Test plan
./format.sh --check(mypy/ruff passed locally; yamlfmt step failed locally due to an unrelated macOS tooling issue, confirmed via directyaml.safe_loadvalidation of edited manifests)🤖 Generated with Claude Code