Skip to content

op-service: make the RPC pre-flight check proxy-aware - #529

Open
shenkeyao wants to merge 1 commit into
espresso/batcherfrom
espresso/proxy-aware-url-check
Open

shenkeyao wants to merge 1 commit into
espresso/batcherfrom
espresso/proxy-aware-url-check

Conversation

@shenkeyao

Copy link
Copy Markdown
Collaborator

Carries cc14081134 from EspressoSystems#2 (branch espresso/proxy-aware-url-check) onto espresso/batcher here, where it belongs: op-service/client/rpc.go is byte-identical in both repos, and EspressoSystems/optimism:espresso/batcher is celo-org's head plus the enclave-packaging commit — anything else carried there is erased by the next fork sync.

The bug

IsURLAvailable pre-flights every RPC endpoint with a raw net.Dial that ignores HTTP_PROXY/HTTPS_PROXY — a route the RPC client never takes. In an AWS Nitro enclave there is no direct route, so the probe fails and CheckAndDial gives up before rpc.DialOptions, which would have succeeded:

crit Application failed  failed to setup: failed to dial L1 RPC: operation failed permanently after 30 attempts: address unavailable

It is not one bad address: every URL left on the proxy path fails the probe. URLs the enclave entrypoint rewrites to a local socat listener still pass, because something really is listening there.

The fix

When a proxy is configured for the URL, probe the proxy's address instead of the target's. No behaviour change without a proxy. Port defaulting moves into a hostPort helper so it covers the proxy URL too, keeping the fail-open for schemes with no well-known port. The environment lookup is indirected through a package var because net/http reads proxy env once per process, which a test cannot undo. No new dependencies.

Why it surfaced now

Not the upstream rebase. The probe predates ethereum-optimism#13146 and ethereum-optimism#16864git log -S 'IsURLAvailable(ctx, addr)' -- op-service/client/rpc.go points at ethereum-optimism#10696; ethereum-optimism#13146 only extracted the existing gate into CheckAndDial and ethereum-optimism#16864 only threaded a connectTimeout through it. celo-rebase-17 and celo-rebase-18 carry a byte-identical function, so no celo rebase introduced it either.

What changed is the enclave entrypoint, in optimism-espresso-integration b23b8aba3f ("Support Sepolia Devnet with TEE (#288)", 2025-12-08). Before it, op-batcher/enclave-entrypoint.bash socat-proxied every URL in URL_ARG_RE to a 127.0.0.1 listener and ran unset http_proxy HTTP_PROXY https_proxy HTTPS_PROXY — the probe always found a live local listener and there was no proxy for it to be blind to. #288 exports HTTPS_PROXY/HTTP_PROXY and adds needs_socat_proxy(), socating only localhost/127.0.0.1/::1/host and leaving external URLs untouched, because rewriting a public HTTPS endpoint to a loopback address sends the wrong Host and SNI and breaks TLS validation. That is exactly the fix's precondition: a proxy is configured and the target is not locally reachable.

The 2026-08-18 port of the packaging into EspressoSystems/optimism (173dd6f7f3) copied the script byte-identically, and the entrypoint has been unchanged since 2026-03-20. The remaining trigger is configuration: --l1-eth-rpc pointing at an external erpc takes the untouched proxy branch, where a host-local L1 would have been socat-rewritten and passed the probe. (Unverified — that lives in the task-definition history, not in either repo.)

Evidence

Reviewers cannot reproduce this locally.

Unitgo vet ./op-service/client/ clean, go test ./op-service/client/ ok, go build ./op-batcher/... ok. The new tests cover the enclave case (target unreachable, proxy reachable); both pre-existing IsURLAvailable tests still pass.

Live — image pr-2 on Chaos as task definition :25 (2026-08-31 00:11 UTC), zero failure markers. The enclave reached L1, the KMS signer, the attestation service and the Espresso query service, proved the Nitro attestation, and landed registerBatcher on-chain (tx 0xd5721974…). Safe test: activeIsEspresso() = false, so the fallback batcher kept batching.

Follow-ups once merged

Stretch

A pre-flight that probes a path the client never uses is an upstream bug in a generic code path, not an Espresso quirk — the enclave only made it reachable. Worth proposing to ethereum-optimism once it has run in production.

🤖 Generated with Claude Code

IsURLAvailable probes an endpoint with a raw net.Dial before
rpc.DialOptions is attempted. That dial ignores HTTP_PROXY/HTTPS_PROXY,
so it tests a route the RPC client itself never takes.

Inside an AWS Nitro enclave there is no direct route at all — every
packet leaves through a local proxy — so the probe fails for every
address and CheckAndDial gives up before reaching rpc.DialOptions, which
would have succeeded. The TEE batcher dies during setup:

  failed to setup: failed to dial L1 RPC: operation failed permanently
  after 30 attempts: address unavailable (http://erpc...:4000/...)

Probe whatever the client will actually dial: when a proxy is configured
for the URL, check the proxy's address instead of the target's. With no
proxy, behaviour is unchanged.

Port defaulting moves into a hostPort helper so it applies to the proxy
URL too, keeping the existing fail-open for schemes with no well-known
port. The environment lookup is indirected through a package var because
net/http reads proxy env once per process, which a test cannot undo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d27e49085

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread op-service/client/rpc.go
Comment on lines +209 to 212
if proxyURL, err := proxyForRequest(&http.Request{URL: u}); err == nil && proxyURL != nil {
if proxyAddr := hostPort(proxyURL); proxyAddr != "" {
addr = proxyAddr
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor the HTTP transport supplied in RPC options

When HTTP_PROXY or HTTPS_PROXY is set but the caller passes rpc.WithHTTPClient with a transport that deliberately bypasses the environment proxy through WithGethRPCOptions, this probes the environment proxy and may reject the endpoint before rpc.DialOptions runs, even though the configured client could connect directly. Proxy selection must reflect the effective client transport, or the preflight should not reject such custom-client configurations.

Useful? React with 👍 / 👎.

Comment thread op-service/client/rpc.go
Comment on lines +209 to 212
if proxyURL, err := proxyForRequest(&http.Request{URL: u}); err == nil && proxyURL != nil {
if proxyAddr := hostPort(proxyURL); proxyAddr != "" {
addr = proxyAddr
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Isolate the existing reachability test from proxy lookup

When the test runner has a reachable HTTP_PROXY or HTTPS_PROXY, TestIsURLAvailableNonLocal still expects the nonexistent HTTP(S) domains in op-service/client/dial_test.go:52-55 to be unavailable. This branch now successfully dials the proxy and returns true for those URLs, so the package test fails in the proxy-configured environment this change targets; those direct-reachability assertions need to disable the proxy lookup or adopt proxy-aware expectations.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants