Expose OpenTelemetry metrics shutdown through MeterProvider

[bm1549]

What Does This Do

Adds a Datadog extension to the OpenTelemetry MeterProvider returned by GlobalOpenTelemetry:

MeterProvider meterProvider = GlobalOpenTelemetry.get().getMeterProvider();
DatadogMeterProvider datadogMeterProvider = (DatadogMeterProvider) meterProvider;
CompletableResultCode result = datadogMeterProvider.shutdown().join(10, TimeUnit.SECONDS);

shutdown() performs a final export and stops Datadog's OpenTelemetry metrics export pipeline. Java does not expose a public forceFlush() extension.

The return type follows OpenTelemetry's CompletableResultCode pattern without using CompletableFuture. A timed join() bounds the caller's wait but does not cancel shutdown. Repeated calls observe the first shutdown result through separate result views. Completion is visible to every view before callbacks run, so a blocked callback on one result cannot hold up another caller. Callers also cannot change the stored result.

Periodic exports, the existing internal flush, and final export remain serialized on the metrics executor. A disabled or unavailable pipeline treats shutdown as a successful no-op. Cancellation, submission, export, and cleanup failures complete the result as failed.

Motivation

Short-lived Java processes need a supported way to wait for pending custom metrics before exit. Exposing shutdown from the meter provider keeps the API next to the OpenTelemetry component it controls.

Additional Notes

• The existing void flushMetrics() bridge retains its signature and 2.5-second timeout.
• Lifecycle submissions do not inherit the caller's active tracing context.
• The Node.js companion PR continues to expose both forceFlush() and shutdown().
• The system-tests and documentation companion PRs need Java-specific updates for the shutdown-only provider API.
• Companion PRs: system-tests #7606, Node.js #10023, and documentation #39515.

Tests run:

• ./gradlew :dd-trace-api:test --tests 'datadog.trace.api.metrics.*Test' :dd-trace-api:javadoc :dd-trace-api:spotbugsMain
• ./gradlew :dd-trace-api:jacocoTestCoverageVerification
• ./gradlew :dd-trace-core:test --tests datadog.trace.core.otlp.metrics.OtlpMetricsServiceTest
• ./gradlew :internal-api:test --tests datadog.trace.bootstrap.instrumentation.api.AgentTracerTest :dd-trace-ot:test --tests datadog.opentracing.DDTracerTest
• ./gradlew :dd-java-agent:instrumentation:opentelemetry:opentelemetry-1.47:forkedTest --tests opentelemetry147.metrics.OpenTelemetryMetricsLifecycleForkedTest
• ./gradlew spotlessApply

Contributor Checklist

• Format the title according to the contribution guidelines.
• Assign the required type: and comp:/inst: labels.
• Add focused JUnit 5 tests.
• Include a companion documentation change.
• Use merge queue after approval.

Jira ticket: N/A

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 46.03%
• Overall Coverage: 58.84% (-0.17%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2fd3467 | Docs | View more details | Give us feedback!}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.01 s	14.06 s	[-1.0%; +0.2%] (no difference)
startup:insecure-bank:tracing:Agent	12.95 s	12.99 s	[-1.1%; +0.3%] (no difference)
startup:petclinic:appsec:Agent	17.11 s	16.84 s	[+0.4%; +2.7%] (maybe worse)
startup:petclinic:iast:Agent	17.11 s	17.01 s	[-0.4%; +1.5%] (no difference)
startup:petclinic:profiling:Agent	16.63 s	16.77 s	[-1.8%; +0.1%] (no difference)
startup:petclinic:sca:Agent	17.07 s	16.83 s	[+0.5%; +2.4%] (maybe worse)
startup:petclinic:tracing:Agent	16.14 s	15.64 s	[-1.1%; +7.4%] (no difference)

Commit: 2fd3467c · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[dougqh]

Note from Claude (reviewing on behalf of @dougqh): not a blocker, but worth thinking through before this lands.

Neither OtlpMetricsService.forceFlush() nor shutdown() puts a deadline on the returned CompletableFuture<Boolean>. The only bound in the chain is the transport's own retry policy:

• OtlpHttpSender retries via HttpRetryPolicy.Factory(5, 100, 2.0, true) — up to 5 retries, and the trailing true means InterruptedIOException (covers SocketTimeoutException) is retried, not just hard connection failures.
• Each attempt is bounded by otlp.metrics.timeout, which defaults to 10s.
• Worst case under a degraded (not dead) network — connects but reads stall — that's roughly 6 attempts × ~10s + backoff ≈ 60+ seconds before export()/finishShutdown() returns and the future completes.

For the automatic path (CoreTracer#close()) this is harmless today, since it calls shutdown() fire-and-forget without awaiting the future.

But it cuts against the PR's own motivation: a short-lived job calling OpenTelemetryMetrics.shutdown().join()/.get() to make sure metrics leave before the process exits could block up to ~a minute under a merely-slow network, which is a rough trade for something meant to bound a short-lived job's exit.

Worth considering wrapping the flush/shutdown future with orTimeout(...)/completeOnTimeout(...) (or using a tighter/no-retry policy specifically for the shutdown-triggered final export) so callers of the new public API get a predictable upper bound independent of the transport's retry behavior.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-01T22:04:50.136580Z	1b4200c	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[datadog-official]

The tracer shutdown path starts the final export on a daemon thread but does not wait for it. The DDTracer wrapper also returns unavailable for both new lifecycle calls instead of forwarding them.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 1b4200c · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

dd-trace-java/dd-trace-core/src/main/java/datadog/trace/core/CoreTracer.java

Line 1532 in 1b4200c

OtlpMetricsService.INSTANCE.shutdown();

Await metrics shutdown from CoreTracer.close

When CoreTracer.close() runs, especially from dd-tracer-shutdown-hook, this call now only enqueues finishShutdown and discards its future. The exporter uses a daemon AgentThreadFactory, so the JVM may terminate after the hook returns while the final send is still running; manual close likewise returns before sender.shutdown() completes. The previous implementation closed the sender synchronously, so wait for this future with a bounded timeout before returning.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mabdinur]

@dougqh On the deadline question: the future intentionally has no built-in timeout. Callers can bound their wait with get(timeout, unit), which preserves Java 8 support and does not imply that the in-flight transport was cancelled. CoreTracer.close() now applies a 2.5-second bound.

[mabdinur]

Reviewed the Datadog lifecycle implementation, addressed feedback, and focused tests. CI is green.

[mhlidd]

From Claude:

Terminal singleton kills pipeline for any subsequent tracer. This is a regression the PR introduces, not pre-existing. Old shutdown() was scheduledTask.cancel(); sender.shutdown(); — no latch, and the AgentTaskScheduler was never shut down, so start() could reschedule. The PR adds both the one-way latch and executor termination.

[mhlidd]

w/ Claude:

The use of Boolean here does not match what Node already does. dd-trace-js#10023 uses forceFlush(callback?: (error?: Error) => void), and its not-configured path calls done?.() with no error, aka success, where Java returns false. So the same "OTLP metrics not enabled" deployment reports success on Node and failure on Java, and Boolean also can't separate that from a real export failure.

Should we align w/ the Node implementation here for cross language parity and to differentiate between real export failures and a shutdown?

[mcculls]

Looking at the other places where we check instanceof InternalTracer we usually fall back to a "no-op" behaviour. The equivalent here would be returning completedFuture(true) which would match what Node does.

This also makes sense from a caller's perspective - if the registered tracer doesn't support OTel metrics then this call would truly be a "no-op" and should succeed because it has nothing to do. Whereas returning false makes it look like there's a problem to be investigated...

[mcculls]

I would make these methods non-default, as otherwise you risk forgetting to implement them (as happened here)

There are only a few implementations of this API, and the javadoc clearly states that methods may be added/removed at any time. This would also help resolve some ambiguity of what the default implementation should be - the only remaining place to fix would be NoopTracerAPI and in that case it should be returning true (success) since there is nothing to flush/shutdown, so those methods always succeed.

[mcculls]

As mentioned above, I'd change this to be a "no-op" style method returning completedFuture(true)

[mcculls]

I have concerns about using CompletableFuture in a public tracer API because this is something we trace. We also try to avoid touching anything fork-join related internally - at least before we've had the chance to do some initial transformations, because it ends up loading certain JDK classes before we get a chance to field-inject them.

( basically CompletableFuture refers to the common ForkJoinPool during static initialization, so ideally we'd avoid touching that class too early. )

I see we've already had to resort to setAsyncPropagationEnabled(false) in the updated implementation of the metrics service which shows this is a real concern.

I wonder if we could use another mechanism here that is simpler and more tracer friendly?

[mcculls]

Side note: no OpenTelemetry API currently uses CompletableFuture (or even anything Future related)

[mcculls]

Potential options, in no particular order:

• Return Future (an interface, not an implementation like CompletableFuture)
• Let callers provide a callback - again, callers will need to decide how to use that
• Return our own "completable" type, like OTel's CompletableResultCode
• Make this a blocking method - callers then decide whether to make it async using their own pool

Note if we did return a Future then we should still try to avoid pulling in CompletableFuture behind the scenes if possible - instead we should try to wrap AgentTaskScheduler.Scheduled as a Future which would simplify the rest of the changes here.

[mcculls]

We should avoid using CompletableFuture in a public trace/metrics API

I've suggested some alternatives in other comments

[mcculls]

If OTel metrics are not enabled this is effectively a no-op and should return true, not false

[mcculls]

If OTel metrics are not enabled this is effectively a no-op and should return true, not false

[datadog-official]

A callback on one shutdown result can block completion of later shutdown results. This behavior breaks the repeated-call result contract.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit cd8ab82 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[datadog-official]

One callback can block other shutdown results

A second caller can time out after shutdown finishes, so repeated shutdown calls do not observe the first result.

Assertion details

• Input: Two callers call shutdown before export finishes. The first caller adds a callback that does not return.
• Expected: The service must complete all shutdown results after export and cleanup finish. One caller callback must not block another result.
• Actual: The first result callback runs on the completion thread. If it does not return, the service does not complete the second result.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest · Open Bits AI session}