Conversation
Re-resolve the six edgezero-* deps from 908e229a to 5f3d648c (current tip of feature/edgezero-deploy-actions, PR #316). The upstream change is an internal review-addressing pass (redact config-store errors, fix version parse, log cleanup, docs) confined to the Fastly adapter CLI; no ts CLI surface change, so no run.rs adaptation is needed.
aram356
marked this pull request as draft
August 16, 2026 19:40
The CLI still called `resolved_gam_unit_path`, which core replaced with the
path-aware `render_gam_unit_path` when `{section}` templating landed, so the
crate no longer compiled. Both call sites now derive the section through
`CreativeOpportunitiesConfig::section_for_path` and render the template, and
`ExpectedSlot`/`ConfiguredJson` carry an optional unit path so an over-limit
dynamic render is reported rather than silently matched against the wrong unit.
Also resolves the outstanding review findings on these paths:
- Write the operator config through a same-directory temp file, fsync, and
rename, so a failed write cannot truncate `trusted-server.toml`.
- Validate TLS certificates in both audit browser sessions; opting out now
requires `--danger-accept-invalid-certs`.
- Refuse a redirect that leaves the requested origin during verify unless
`--allow-cross-origin-redirect` is passed, so another origin's evidence
cannot satisfy `--strict`.
- Reject page patterns the runtime cannot compile before they reach the file,
through a new shared `compile_page_pattern` in core.
- Reject `creative_opportunities` declared in a form the line-based splice
cannot edit, instead of appending a duplicate table.
- Drop non-integer GPT sizes in the collector so one fluid size cannot fail
deserialization of the whole evidence payload.
- Escape control characters in page-controlled text written to the terminal.
`ts audit ad-templates generate` derived everything it wrote from a live,
page-controlled ad stack and never checked the result, so several reachable
inputs produced a config that cannot load. An unloadable `trusted-server.toml`
is not a degraded ad stack: `build_state` fails and the adapter answers every
route from the startup error router, so the whole site returns 500 once pushed.
Add a write-side gate that runs the candidate through `Settings::from_toml`,
the same `finalize_deserialized` chain the runtime uses at startup. It runs on
the `--dry-run` path too, so a clean preview is now evidence the config loads.
When the target config was already unloadable before the run, the gate reports
that as a warning instead of blaming this run, so a freshly bootstrapped file
carrying placeholder secrets can still be updated.
Close the three reachable paths at their source as well:
- Skip a scraped slot whose ad-unit path contains `{` or `}`. The path is a
template and there is no escape syntax, so a literal brace either fails
config load or is silently reinterpreted as a placeholder.
- Skip a slot whose div id normalizes to nothing (a wholly ephemeral id such
as a React SSR marker). An empty `div_id` fails config load, and as a runtime
prefix it would bind the slot to the first id-bearing element on the page.
- Refuse to create a `[creative_opportunities]` section with no GAM network id
rather than writing one that omits the required key. This is reachable
because the network id is only recovered from an all-digit leading segment,
which an MCM child-network path does not have.
Groundwork for discovering ad slots across a site's sections rather than from a single page. Nothing calls this yet; `run_update_slots` is unchanged. `AuditCollector` gains a defaulted `collect_pages` that streams each page to a sink, so every existing implementor keeps working and the caller can fold a page into its evidence and drop the DOM immediately instead of holding every serialization at once. The browser collector overrides it to launch Chrome once for the whole crawl: a cold start plus a fresh profile dominates the cost of a multi-page run, and the shared profile carries a bot-protection clearance cookie earned on the first page across the rest of the walk. Page discovery reads the hydrated DOM rather than the served markup, because an app-router page keeps its link graph in the framework payload — parsing raw HTML finds only a fraction of a site's sections. Sitemaps are fetched from inside the open page via `fetch` plus `DOMParser`, which inherits the session's cookies and Chrome's TLS fingerprint, gets transparent gzip and XML parsing, and so needs no new Rust dependency. `crawl_plan` turns links and sitemap entries into a bounded page set: one landing page and one article per section, ranked by whether navigation and the sitemap corroborate each other, capped by section and page budgets. Sections dropped for budget are reported rather than silently omitted. Same-origin is enforced on links and on sitemap entries alike, since a `Sitemap:` directive can name any host and the crawl carries operator cookies.
Template inference needs the set of observations per slot, not one snapshot: a single page cannot distinguish a literal ad-unit path from a templated one, so the divergence across pages is the only signal available. Add the table that holds it. Nothing calls this yet. Slots are keyed on the normalized div stem, since raw GPT div ids carry per-render framework hashes and would otherwise look like a new slot on every page. Three reconciliations happen here and nowhere else: - Formats union across pages. A size that renders only on article pages, such as a 300x600 rail, has to survive alongside the homepage's sizes; taking the first page's list would silently narrow the slot. - Divergent unit paths are retained as separate rows rather than collapsed, because discarding them is what makes templating impossible. - Network ids must agree. Two GAM networks in one crawl means the pages are not one property, so this is a hard error naming both rather than a guess that would bid against the wrong inventory. Pages that yield no slots are recorded rather than dropped, so a caller can recognise a bot challenge serving interstitials and refuse to write a half-empty config.
Adds the inference that turns literal scraped ad-unit paths into a
`{network_id}`/`{section}` template plus the section policy it depends on.
Nothing calls this yet.
A wrong template makes a publisher bid against inventory that does not exist,
which is worse than a narrow literal path, so this refuses rather than guesses.
Three rules carry that:
- `{network_id}` binds positionally to unit segment 0 and only when that segment
already equals the resolved id. Substring replacement would rewrite
`/123/sports123/home` into `/{network_id}/sports{network_id}/home`.
- Exactly one unit segment may vary. Zero proves nothing and stays literal; two
means the unit tracks a dimension the request path cannot supply, such as a
device or geo split, and is refused with that reason.
- Two pages must witness both a different derived section and a different unit
segment before anything is templated. Round-trip verification cannot supply
this: a single observation is reproduced equally well by a literal path, a
`{network_id}`-only template, and a `{section}` template, so only variation
distinguishes them.
`section_segment` is chosen by partitioning observations into pages that have a
section segment and pages that do not, the latter fixing `section_root`. An
index that cannot be witnessed is rejected, an unwitnessed root leaves the path
literal rather than guessing, and two indices that both fit are ambiguous and
template nothing.
Every accepted template is then replayed through the runtime's own
`render_gam_unit_path` and `derive_section` against every observation, so a
section slug the path cannot reproduce is caught and downgraded. `derive_section`
becomes public for exactly this: the check has to use the runtime's derivation
rather than a second implementation that could drift from it.
…terns Two gaps between what inference produces and what the writer could put on disk. Nothing calls the new code yet. `page_patterns` expands the paths a slot was observed on into globs. Each witnessed section contributes a pair, because one glob cannot cover both halves: `*` crosses `/` in this dialect, so `/news/*` matches `/news/a/b` but not the bare `/news` landing page, and emitting only the star form would silently drop the landing page from the slot. Nothing extrapolates past a witnessed section, so a crawl that never visited `/reviews` never claims it. `replace_key_in_section` can only rewrite a key that is already present, so it could not add `section_root` or `section_segment` to a config that predates them, which is every config a first templated run touches. Add `upsert_key_in_section`, which inserts immediately after the section header so the new key lands in the section's scalar block rather than after a subtable, where TOML would read it as belonging to that subtable instead. `splice_creative_slots` now takes the section keys as a struct rather than a bare network id. It omits `section_root` and `section_segment` entirely unless a slot actually templated: both are `deny_unknown_fields` additions, so writing them into a config that does not need them would make it unloadable by an older binary for no benefit.
Connects the crawl, evidence, inference and writer pieces: a bare
`ts audit ad-templates generate <url>` now samples the site's sections,
reconciles each slot across them, infers a `{section}` ad-unit template where
the evidence proves one, and writes the section policy alongside the slots.
The flow is collect root, plan the crawl from its links and sitemap, walk the
planned pages on one browser, fold each into the evidence table, infer, then
merge, render, splice and validate as before. Page patterns now come from the
sections a slot was actually seen on, so a slot scraped from one article serves
its whole section instead of that single URL.
Failure handling follows what the evidence can support. A page that will not
collect is reported and skipped, because one blocked page should not discard
the sections that worked. But if more than a quarter of crawled pages yield no
slots the run refuses outright: that is the signature of bot protection serving
challenge interstitials, and writing from it would silently narrow the
operator's slot set. Pages disagreeing about the GAM network id is likewise a
refusal rather than a guess.
A run that templates prints the deploy-ordering contract, because the config it
just wrote is not rollback-safe: `section_root` and `section_segment` are
`deny_unknown_fields` additions, so an older binary rejects the whole config and
serves an error on every route.
`--max-pages` and `--max-sections` bound the crawl; `--max-pages 1` restores
single-page behavior exactly, and an explicit `--page-pattern` still applies to
every slot and skips pattern inference. `run_update_slots` takes a request
struct, since a nine-argument signature could not absorb the crawl bounds.
Removes `default_page_pattern`, superseded by section-derived patterns, and
narrows the single-page `merge_slots` path to test scaffolding.
Publishers routinely serve a different GAM ad unit per device (`/network/desktop/news` against `/network/mobile/news`). A single-profile crawl cannot see that: it infers a template that is correct for the profile it used and silently wrong for every other impression, with nothing in the data to say so. This was the one unmitigated risk in the inference design. `--profiles desktop,mobile` walks every planned page once per profile, each with its own viewport and user agent, folding all of it into one evidence table. The user agent matters as much as the viewport here — ad stacks branch on it, so emulating size alone can still return desktop ad units on a phone-sized page. No new refusal logic was needed. Two profiles disagreeing produce two ad-unit paths for a single page, which is already the structural refusal inference applies to a unit that varies by something the request path cannot derive. The slot is still written, with its div and formats intact, but with no `gam_unit_path`: no path at all is better than one that is wrong on mobile, and the runtime falls back to the default unit rather than bidding on a unit that does not exist. Desktop-only stays the default, so the extra crawl is opt-in.
`ts audit ad-templates generate` had no documentation at all. Cover what the crawl does, what it writes, and the two things an operator cannot discover from the output alone. The first is when the command declines to generalize. A wrong ad-unit template makes a publisher bid against inventory that does not exist, so the command prefers a narrow literal path over a plausible guess, and the table says which situations produce which outcome — including the cases that fail the run outright, such as a crawl where bot protection served mostly challenge pages. The second is deploy ordering. A config carrying `section_root` or `section_segment` is not rollback-safe: a binary predating ad-unit templating rejects those keys, and the rejection fails the whole configuration load rather than just the ad-template section, so every route serves an error. Ship the template-aware binary first, push second, and do not roll back while that config is live.
A live run against a bot-protected site refused with "no slots discovered" and nothing else, because the per-page reasons were collected and then thrown away: `fold_collected` discarded each page's collector warnings, and both refusal paths returned before any note was printed. The guards exist for runs that went wrong, so that is exactly when the reasons matter. Notes are now drained as soon as the crawl finishes, ahead of the refusals, and each page's warnings are attributed to its path. Also name the failure that has no warning of its own. Bot protection commonly answers with 200 and a challenge document rather than a 4xx, so the status check passes, the page settles cleanly, and it simply appears to run no ad stack — indistinguishable from a publisher who genuinely has none, though the operator's next move differs completely. A page carrying almost no scripts and no recognised integrations is now called out as a probable challenge, with the advice to supply a current cookie. Verified against a live protected origin: the run previously reported only that no slots were found; it now identifies the interstitial and says what to do.
A live crawl of a bot-protected origin returned the real page for the first request and a challenge for the remaining thirteen. A dead cookie fails on the first page, so that shape points at the session being flagged during the run rather than at the credential. Two contributors, both worth correcting regardless of that diagnosis. The crawl issued its navigations back to back. That is discourteous to the origin on its own terms, and request pacing is among the signals bot protection scores, so an unpaced crawl invites the challenge that empties the rest of the run. `--page-delay-ms` now spaces them, defaulting to 750ms. Headless Chrome is trivially detectable, so an origin that serves the real page to a normal browser may answer the same request headless with a challenge. `--headful` runs a visible browser for the cases where that is the difference. Note that `BrowserConfig` defaults to the *old* headless mode, so simply not requesting new-headless yields a more detectable browser rather than a headful one. Both branches are explicit for that reason.
A live audit of a consent-gated publisher reported no ad slots and gave no way to tell why. Two additions, found by debugging exactly that. Publishers gate slot definition behind their consent platform, and a fresh audit profile has no consent cookie, so the crawl never reaches `googletag.defineSlot` and the page looks like it has no ad stack at all. The audit browser now answers the two IAB interfaces every compliant platform exposes, TCF v2 and US Privacy, installed before any page script runs so the real platform finds them already defined. `gdprApplies: false` avoids fabricating a consent string and matches the signal genuinely out-of-scope traffic carries. `--no-assume-consent` observes the un-consented page instead. When the slot registry comes back empty, the run now reports what GPT actually looked like — whether the library reached `apiReady`, how many queued commands never drained, whether `pubads()` exists, and how many scripts the page ran. An empty registry has several very different causes, and the operator's next move differs for each. Against a local proxy this immediately distinguished "GPT never finished loading" from "this page has no ads", which no amount of re-running could have shown before.
Verified against a live publisher served by `ts dev proxy`, which surfaced two
defects that no fixture could.
`normalize_div_stem` matched only the uppercase React `_R_` marker. React also
emits the lowercase `_r_0_` form client-side, and the token changes on every
render, so a slot arrived as `ad-header-0-_r_0_` on one page and
`ad-header-0-_r_8_` on the next. One logical slot fragmented into a new key per
page: the written `div_id` would never match at runtime, and template inference
saw no slot twice, so it had no variation to reason about and kept every path
literal. Collapsing the lowercase form is what lets the crawl rediscover
`/{network_id}/autoblog/{section}` from live evidence.
Add `--browser-proxy` so the audit can run against a production hostname served
locally, which keeps the page's origin, cookie scope, and any origin checks in
the ad stack matching production rather than `localhost`. `--danger-accept-
invalid-certs` covers a MITM certificate whose CA the throwaway browser profile
does not trust.
Note that chromiumoxide builds each Chrome flag by prefixing `--` to the arg
key, so a pre-formatted `--flag=value` string becomes `----flag=value` and is
silently dropped. Both the new proxy flags and the existing mobile user-agent
override were written that way; the user-agent override had therefore never
taken effect. Both now pass `(key, value)` pairs.
A live crawl produced fourteen slots where four were real. Ten were two placements repeated: an ad stack built its div ids from a per-render token, so the same placement arrived under a new key on every page. Written verbatim those ids match nothing at runtime, and the fragmentation also starves template inference, which needs to observe a slot more than once. Detect it from evidence rather than by pattern-matching token shapes, since each stack invents its own and the previous two forms already needed separate handling. Candidates share an identical ad-unit path and identical formats; what separates a fragmented placement from two legitimate siblings on one unit is co-occurrence. Real siblings appear together on a page, while fragments never do, because each page yields exactly one of them. Fragments are reported and skipped rather than written. The report names the observed ids and the stable prefix they share, so the operator can add the placement once with a prefix they know survives a render. That prefix is deliberately not written as a `div_id`: it reaches only as far as the observed tokens happen to agree, so it would match this crawl's ids and miss the next render's. Verified live: the run that previously wrote fourteen slots now writes the four real ones and explains the two it declined.
Four options landed after the command was first documented and were never written up: request pacing, a headful browser, the consent answer, and auditing through a local proxy. Each exists because a live audit of a protected publisher failed without it, so the reason belongs alongside the flag. Consent gets its own section because the failure is silent. A publisher gates slot definition behind its consent platform, the audit runs in a throwaway profile with no consent cookie, and the result is a page that appears to have no ad stack at all — indistinguishable from one that genuinely has none. Also record that an empty slot registry now reports GPT's observable state, which is what separates "the library never loaded" from "this page has no ads". Proxy auditing gets a section because `ts dev proxy` is how a production hostname is served locally, and matching the production origin matters for cookie scope and for origin checks inside the ad stack. Note the caveat that a local Trusted Server injects its own configured slots, so a run through the proxy can rediscover config it already has. Finally, describe how per-render div ids are detected and reported, including why the suggested prefix is offered but never written.
Resolve route-registration conflicts in the axum, spin, and fastly adapters.
Main's `/__ts/page-bids` legacy alias landed next to the admin EC lookup
routes, so the named-route array lengths and a shared `#[test]` attribute
collided:
- axum `named_routes` and spin `named_fallback_paths` now declare 16 entries
(13 from main plus `/_ts/admin/ec`, `/_ts/admin/ec/{id}`, `/_ts/admin/eids`).
- fastly keeps both colliding tests: `admin_ec_lookup_routes_are_registered`
and `page_bids_serves_canonical_path_and_deprecated_alias`.
…nt-config # Conflicts: # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/settings.rs
main squash-merged #928 as 1c1ab84; rc had already merged that PR's implementation branch at 464acd8^, so the squash carried exactly one new commit — 464acd8, hardening the admin fallback reservation and the bare-route cookie resolution. Resolutions: take main's ec/admin.rs whole (rc's blob was byte-identical to 464acd8^, so nothing rc-only was at stake), take only the theirs side of the adapter route-test hunks so the rc-only TSJS cache-header tests survive, keep rc's Changed entry for the inactive-template cache policy, and take main's newer wording for the admin-namespace Security entry. The merge delta is byte-identical to 464acd8.
# Conflicts: # crates/trusted-server-adapter-fastly/src/main.rs # crates/trusted-server-core/src/integrations/gpt_diagnostics.rs # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/response_privacy.rs # crates/trusted-server-core/src/settings.rs # crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/badges.ts # crates/trusted-server-js/lib/test/integrations/gpt/ad_init.test.ts # crates/trusted-server-js/lib/test/integrations/gpt_diagnostics/api.test.ts # crates/trusted-server-js/lib/test/integrations/gpt_diagnostics/badges.test.ts # docs/guide/configuration.md # docs/guide/integrations/gpt-diagnostics.md
# Conflicts: # docs/guide/cli.md
# Conflicts: # CHANGELOG.md # crates/trusted-server-adapter-fastly/src/middleware.rs # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/response_privacy.rs
Resolve ten conflicts from main's three commits after the #860 merge base. #943 and #1013 reached main as squashes after rc had already merged earlier revisions of the same branches, and #1066 is new to rc. Take main where its commit is the newer revision: - Fastly `apply_terminal_response_effects`: `must_remain_private` narrows back to the `TerminalPrivateResponse` marker, so an unmarked origin `private, max-age=600` keeps its own policy and validators instead of being overwritten. The terminal guard runs both `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`. Adds main's three review tests for page-bids preflight, active diagnostics, and the unmarked-origin case, while keeping rc's `resolve_and_sanitize_client_ip` block. - The three `docs/superpowers` documents: #1066's dead-link fixes, with no other difference between the two versions. Keep rc where it is the functional superset, after confirming no line main added is missing from it: - `response_privacy.rs` keeps #1008's inactive-ad-stack browser policy, the quoted-string-aware directive parser, and the public `is_private_or_no_store`; main's two new edge-cache-header tests are grafted on. - `publisher.rs` keeps `PageBidsFormat::parse`, which supersedes main's inline `format` rejection, and the assembled-response privacy condition that already covers main's narrower form. - `creative_opportunities.rs` and `scripts/template-cache-local-test.sh`. Two artifacts of the automatic merge needed correcting: `settings.rs` gained ten duplicate `#[test]` definitions from the #943 squash meeting rc's native merge, which `cargo check` cannot see because it does not build tests, and `configuration.md` had a duplicated shared-template heading. `Cache-Control` keeps rc's `private, no-store` ordering. #1013's final revision flipped the `enforce_private_no_store` path to `no-store, private`; directive order carries no meaning, rc asserts the current spelling in 34 places, and the CI shell harness greps the literal string. Main's grafted assertions were adapted instead. The JA4 debug endpoint keeps its own `no-store, private` literal.
Unify Tinybird, DataDome, and S3 static credentials under the logical default secret store, resolve them during typed config loading, and remove request-time static secret reads. Honor Fastly logical-to-physical store mappings, preserve deserialize-only selector compatibility, redact runtime values, and document provisioning and migration behavior.
# Conflicts: # Cargo.lock # Cargo.toml # crates/trusted-server-adapter-fastly/src/app.rs # crates/trusted-server-adapter-spin/src/app.rs # crates/trusted-server-core/src/config.rs # crates/trusted-server-core/src/integrations/datadome.rs # crates/trusted-server-core/src/integrations/datadome/protection.rs # crates/trusted-server-core/src/settings.rs # docs/guide/configuration.md # docs/guide/integrations/datadome.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
August release candidate: current
mainplus the remaining open PRs, staged together for RC validation. Supersedes the July RC (#919).Already included through main:
In Review:
Verification
cargo fmt --all -- --check-D warningscargo test-fastly(2,075 tests via Viceroy),test-axum,test-cloudflare,test-spin./scripts/test-cli.sh(34 tests, exercising the Upgrade EdgeZero to the deploy-actions branch #940run.rschanges)