From 4bb56f9f88e1a481430bdb08a7826664b9445840 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:29:58 -0700 Subject: [PATCH 01/13] Add Prebid ad-latency optimizations design spec Three Trusted-Server-side levers for client-side auction properties, grounded in an instrumented local baseline against the autoblog origin: an opt-in requestBids coalescing window, a securepubads preconnect hint, and an opt-in first-refresh-auction prefetch at DOMContentLoaded that keeps the React-hydration-safe DOM application point unchanged. --- ...-prebid-ad-latency-optimizations-design.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md new file mode 100644 index 000000000..f969584c0 --- /dev/null +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -0,0 +1,98 @@ +# Prebid Ad-Latency Optimizations — Design + +**Date:** 2026-08-20 +**Status:** Approved design, pending implementation plan +**Scope:** Client-side auction properties (server-side ad templates inactive) + +## Problem + +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ads render slowly relative to what the pipeline allows. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: + +| Milestone | Time | +| --------------------------------------- | ------ | +| Prebid bundle + shim fully installed | ~2.1 s | +| `DOMContentLoaded` | ~2.1 s | +| `window.load` | ~3.0 s | +| First Trusted Server `/auction` request | ~3.3 s | +| Publisher's first `requestBids` | ~4.8 s | +| First non-empty ad render | ~5.0 s | + +Three structural costs stand out, all in Trusted-Server-owned surfaces: + +1. **The first refresh auction waits for `window.load`.** The post-`load` + double-`requestAnimationFrame` defer exists to avoid React hydration mismatches (React #418), but only the DOM application needs that defer — the auction network fetch is hydration-neutral. `DOMContentLoaded` fires ~1.2 s before `load` on this page, and the gap grows on resource-heavy pages. +2. **Publisher `requestBids` bursts pay one `/auction` round trip each.** The baseline captured two `requestBids` calls 1 ms apart producing two `/auction` POSTs 2 ms apart. Each call costs a full round trip plus ~865 ms median (p90 ~1075 ms) of server-side auction time. +3. **The first GAM ad request pays fresh connection setup.** `securepubads.g.doubleclick.net` serves both the `pubads_impl` script and every ad request; no connection is warmed before first use. + +Not addressed here (out of scope): re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency (~1.8 s after `load` before their first `requestBids`), and server-side auction duration tuning (PBS `tmax`). + +## Design + +Three independent levers. Two are config-gated and default off, so shipping the binary changes nothing until an operator flips the flag; one is an unconditional resource hint. + +### Lever A — `requestBids` coalescing window (opt-in) + +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=500`. Serialized into the injected client config (`window.__tsjs_prebid`) as `requestBidsCoalesceMs`, omitted when `0`. + +**Behavior:** The tsjs Prebid shim already wraps `pbjs.requestBids` (bidder injection, snapshot capture, `bidsBackHandler` chaining). With a non-zero window, the wrapper holds a transformed call for up to the window duration and merges every mergeable call that arrives within it into one underlying `requestBids`: + +- **Merged request:** union of the pending calls' ad units (in arrival order), maximum of their `timeout`s (absent if none supplied), and a combined `bidsBackHandler` that invokes each pending call's (already-wrapped) handler in arrival order with the merged auction's results. A throwing handler must not prevent later handlers from running. +- **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. +- **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. + +**Return value:** held calls return `undefined` from `pbjs.requestBids`. This matches the wrapper's existing contract in practice, and the flag's opt-in nature means an operator enabling it accepts this for their property. + +### Lever B — GAM preconnect hint + +The GPT integration's `head_inserts` emits, before its bootstrap scripts: + +```html + +``` + +Unconditional: it is a pure hint with no behavioral effect, and every GPT-enabled property talks to this host. Removes DNS + TCP + TLS setup from the first ad request (and benefits the `pubads_impl` fetch). + +### Lever C — first-auction prefetch at `DOMContentLoaded` (opt-in) + +**Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. + +**Behavior:** When enabled, the first refresh auction's `/auction` request is dispatched as soon as all of the following hold, without waiting for `window.load`: + +- consent has resolved (the existing consent gate is unchanged), +- the GPT slots that the auction would target have been observed, +- `DOMContentLoaded` has fired. + +The response is held in memory and applied at the **unchanged** post-`load` + double-`rAF` application point. DOM work therefore keeps the exact hydration-safety timing the React #418 fix established; only the network round trip moves earlier. + +**Fallback:** if the prefetched response has not arrived by the time the application point runs, the path degrades to today's behavior (issue the request then). A prefetch failure is discarded and the normal path retries; no new error surface. + +## Config-blob compatibility + +Both new fields serialize only at non-default values, matching the repository's rollback discipline: a blob that never sets them is accepted by older binaries, and before rolling a binary back past this feature, an operator must clear the flags and re-push (the same procedure documented for prior `[integrations.prebid]` additions). + +## Testing + +**Vitest (shim):** + +- coalescing merges two mergeable calls in the window into one underlying `requestBids` with the union of ad units, and invokes both handlers in order; +- a call with extra option keys flushes the queue first and dispatches solo, preserving order; +- a throwing first handler does not prevent the second handler from running; +- window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); +- prefetch: response held and applied at the application point; unresolved prefetch falls back to the current request-then-apply path. + +**Rust:** + +- config defaults (`request_bids_coalesce_ms = 0`, `prefetch_first_refresh_auction = false`) and the `0..=500` validation bound; +- injected-config serialization omits both fields at their defaults and includes them otherwise; +- GPT `head_inserts` contains the preconnect link. + +**End-to-end verification:** re-run the local baseline harness (Viceroy proxying the production origin, pre-seeded consent, identical scroll script) with each flag enabled, comparing against the recorded baseline: time to first non-empty render, `/auction` request count, and burst dedup on the publisher's paired calls. + +## Rollout + +1. Land binary with defaults off; the preconnect hint is the only immediate change. +2. Enable `request_bids_coalesce_ms` (initially 50 ms) and `prefetch_first_refresh_auction` on the autoblog tester property via config push. +3. Compare live tester metrics with the harness prediction; each flag reverts independently by config push, no binary rollback. From 796da051d7aeb054313bc556dc4cafaa31acf425 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:36:34 -0700 Subject: [PATCH 02/13] Correct spec: slim-prebid loader gate and requestBids promise contract Lever C: the window.load gate is installSlimPrebidLoader (the auction module is not loaded before load), so the prefetch lives in the unified GPT bundle and slim-Prebid consumes the stashed response; the double-rAF defer belongs to the SSAT adInit path, not this flow. Lever A: Prebid 10 requestBids returns a promise, so held calls return a promise settling with the merged auction instead of undefined. --- ...20-prebid-ad-latency-optimizations-design.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index f969584c0..73910933f 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -39,7 +39,7 @@ Three independent levers. Two are config-gated and default off, so shipping the - **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. - **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. -**Return value:** held calls return `undefined` from `pbjs.requestBids`. This matches the wrapper's existing contract in practice, and the flag's opt-in nature means an operator enabling it accepts this for their property. +**Return value:** Prebid 10's `requestBids` returns a promise, and the wrapper currently passes it through. Every held call therefore returns a promise that settles when the merged underlying `requestBids` promise settles, so callers awaiting the promise keep working; they observe the merged auction's completion rather than a per-call auction's. ### Lever B — GAM preconnect hint @@ -59,15 +59,19 @@ Unconditional: it is a pure hint with no behavioral effect, and every GPT-enable **Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. -**Behavior:** When enabled, the first refresh auction's `/auction` request is dispatched as soon as all of the following hold, without waiting for `window.load`: +**Mechanism today:** the `window.load` gate is `installSlimPrebidLoader` in the unified GPT bundle — the slim-Prebid module that runs refresh auctions is not even _loaded_ until `load` fires, and its first auction follows its post-load initialization. (The post-`load` + double-`rAF` defer from the React #418 fix belongs to the server-side-template `adInit` path and is not part of this flow.) + +**Behavior when enabled:** the unified GPT bundle — which is loaded from head-start — dispatches the first refresh auction's `/auction` request as soon as all of the following hold, without waiting for `window.load`: - consent has resolved (the existing consent gate is unchanged), -- the GPT slots that the auction would target have been observed, +- the GPT slots the auction would target have been observed, - `DOMContentLoaded` has fired. -The response is held in memory and applied at the **unchanged** post-`load` + double-`rAF` application point. DOM work therefore keeps the exact hydration-safety timing the React #418 fix established; only the network round trip moves earlier. +The response is stashed on `window.tsjs`. The slim-Prebid module, still loaded at the unchanged `window.load` point, consumes the stash during its existing initialization instead of issuing a fresh request. Only the network round trip moves earlier; script loading, targeting application, and the GPT refresh that triggers rendering keep their current post-load timing, so no render work moves into the hydration window. + +**Alternative considered and rejected:** loading the slim-Prebid script itself at `DOMContentLoaded`. Simpler, but it would also pull GPT refresh — and therefore ad rendering into publisher containers — earlier into the hydration window, the same territory as React #418 and the ad-container hydration gating tracked in #969. -**Fallback:** if the prefetched response has not arrived by the time the application point runs, the path degrades to today's behavior (issue the request then). A prefetch failure is discarded and the normal path retries; no new error surface. +**Fallback:** if the stash is absent or its request has not resolved when slim-Prebid initializes, the module issues its own request exactly as today. A failed prefetch is discarded; no new error surface. ## Config-blob compatibility @@ -81,7 +85,8 @@ Both new fields serialize only at non-default values, matching the repository's - a call with extra option keys flushes the queue first and dispatches solo, preserving order; - a throwing first handler does not prevent the second handler from running; - window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); -- prefetch: response held and applied at the application point; unresolved prefetch falls back to the current request-then-apply path. +- prefetch: a stashed response is consumed by slim-Prebid initialization without a second `/auction` request; an absent or unresolved stash falls back to the current request path; +- held `requestBids` calls return a promise that settles with the merged auction. **Rust:** From 63fb2211e3439ba8b1dfcf9a948d129dd14b6a86 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:53:15 -0700 Subject: [PATCH 03/13] Return spec to draft and address review findings Reframe Lever A as a load/QPS lever (measured burst POSTs were already concurrent) with absolute deadlines, per-caller bid-map partitioning, bookkeeping-before-callbacks ordering, synthetic-refresh and client-side-bidder exclusions, disjoint-code and payload-size admission bounds, and full promise semantics. Gate Lever B behind [integrations.gpt] gam_preconnect, correct the pubads_impl and crossorigin claims, and note the pre-consent connection decision. Demote Lever C to discovery-first: the described lifecycle seam does not exist, responses cannot be stashed without the requesting auction's bid-request IDs, consent readiness needs a concrete API contract that distinguishes consent-denied no-bids, and fallbacks must share the in-flight promise rather than duplicate auctions. Add a billing and impression integrity section, correct config-blob compatibility to the raw-JSON retention layer, and expand testing, measurement methodology, and one-lever-at-a-time rollout. --- ...-prebid-ad-latency-optimizations-design.md | 151 +++++++++++------- 1 file changed, 92 insertions(+), 59 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 73910933f..83d69a77f 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,103 +1,136 @@ -# Prebid Ad-Latency Optimizations — Design +# Prebid Ad-Latency and Auction-Load Optimizations — Design **Date:** 2026-08-20 -**Status:** Approved design, pending implementation plan +**Status:** Draft (returned from review; Lever C requires a discovery phase before implementation) **Scope:** Client-side auction properties (server-side ad templates inactive) ## Problem -On properties running the client-side auction path (`creative_opportunities.enabled = false`), ads render slowly relative to what the pipeline allows. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: -| Milestone | Time | -| --------------------------------------- | ------ | -| Prebid bundle + shim fully installed | ~2.1 s | -| `DOMContentLoaded` | ~2.1 s | -| `window.load` | ~3.0 s | -| First Trusted Server `/auction` request | ~3.3 s | -| Publisher's first `requestBids` | ~4.8 s | -| First non-empty ad render | ~5.0 s | +| Milestone | Time | +| -------------------------------------------------------------------- | ------ | +| Prebid bundle + shim installed (deferred head scripts, execute ~DCL) | ~2.1 s | +| `DOMContentLoaded` | ~2.1 s | +| `window.load` | ~3.0 s | +| First Trusted Server `/auction` request | ~3.3 s | +| Publisher's first `requestBids` | ~4.8 s | +| First non-empty ad render | ~5.0 s | -Three structural costs stand out, all in Trusted-Server-owned surfaces: +Observed costs, each owned by a different lever below: -1. **The first refresh auction waits for `window.load`.** The post-`load` + double-`requestAnimationFrame` defer exists to avoid React hydration mismatches (React #418), but only the DOM application needs that defer — the auction network fetch is hydration-neutral. `DOMContentLoaded` fires ~1.2 s before `load` on this page, and the gap grows on resource-heavy pages. -2. **Publisher `requestBids` bursts pay one `/auction` round trip each.** The baseline captured two `requestBids` calls 1 ms apart producing two `/auction` POSTs 2 ms apart. Each call costs a full round trip plus ~865 ms median (p90 ~1075 ms) of server-side auction time. -3. **The first GAM ad request pays fresh connection setup.** `securepubads.g.doubleclick.net` serves both the `pubads_impl` script and every ad request; no connection is warmed before first use. +1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`.** The exact trigger of the observed 3.3 s `/auction` request is not yet attributed (it did not pass through `pbjs.requestBids`, and `gpt.slim_prebid_url` is not configured on the measured property). Lever C starts with a discovery task to attribute it precisely. +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a server-load and bidder-QPS cost, not a first-render latency cost. Lever A is therefore a load/cost lever, not a latency lever. +3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. -Not addressed here (out of scope): re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency (~1.8 s after `load` before their first `requestBids`), and server-side auction duration tuning (PBS `tmax`). +Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). + +## Billing and impression integrity (applies to every lever) + +Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see: + +- Prefetched or early-dispatched auctions are **targeting-only**: they produce bids, never renders. `nurl`/`burl` and any render-bridge activity remain tied to an actual GAM render of the slot, exactly as today. +- A prefetched bid that is never consumed expires without firing any beacon. +- Coalescing must not cause a caller's handler to render or fire beacons for another caller's ad units (see the partitioning rule in Lever A). +- Rollout guardrails (below) monitor duplicate-auction rate and beacon counts per rendered impression so any integrity regression is visible immediately. ## Design -Three independent levers. Two are config-gated and default off, so shipping the binary changes nothing until an operator flips the flag; one is an unconditional resource hint. +### Lever A — `requestBids` coalescing window (opt-in; load/cost reduction) + +**Objective:** reduce `/auction` request count and upstream bidder QPS for bursty publisher call patterns. Explicitly **not** claimed to reduce first-render latency; the hold can only add up to the window duration for the earliest caller, and rollout must verify render latency does not regress. + +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected into `window.__tsjs_prebid` as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. + +**Admission (merge-safety) rules.** A call is held only when all of: -### Lever A — `requestBids` coalescing window (opt-in) +- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array (any other key — `ortb2`, `labels`, `adUnitCodes`, `ttlBuffer`, `auctionId`, … — flushes the pending queue synchronously, then dispatches solo, preserving call order); +- it is **not** one of the shim's own synthetic refresh auctions (their GPT watchdog deadline starts when the wrapper returns; holding them races the watchdog and can discard valid bids); +- none of its ad units contains a bid entry for a configured client-side bidder (merging would merge those bidders' native auctions too, changing their request shape and analytics; if operators later want that, it is a separate, explicit decision); +- every held ad-unit `code` is non-empty and **disjoint** from the codes already pending (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types — a merged duplicate would produce a hybrid auction neither caller requested); +- the projected serialized payload of the merged request stays under a bound with safety margin (the endpoint rejects bodies over 256 KiB; bound pending calls, total ad units, and projected bytes — flush before admitting a call that would exceed any bound). -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=500`. Serialized into the injected client config (`window.__tsjs_prebid`) as `requestBidsCoalesceMs`, omitted when `0`. +**Deadlines.** Each call's effective deadline is absolute: queue residence counts against it. A held call with timeout `T` arriving at `t0` must reach Prebid with an adjusted timeout of `T − (dispatch − t0)`. Calls merge only when their absolute deadlines are within a compatibility tolerance; an incompatible deadline flushes the queue. A call without a timeout uses the configured default for this computation. -**Behavior:** The tsjs Prebid shim already wraps `pbjs.requestBids` (bidder injection, snapshot capture, `bidsBackHandler` chaining). With a non-zero window, the wrapper holds a transformed call for up to the window duration and merges every mergeable call that arrives within it into one underlying `requestBids`: +**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the minimum adjusted deadline; and a combined `bidsBackHandler` that: -- **Merged request:** union of the pending calls' ad units (in arrival order), maximum of their `timeout`s (absent if none supplied), and a combined `bidsBackHandler` that invokes each pending call's (already-wrapped) handler in arrival order with the merged auction's results. A throwing handler must not prevent later handlers from running. -- **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. -- **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. +1. first runs **all** Trusted Server bookkeeping for every constituent call (pending-publisher-bid registration, EID cookie sync) — before any publisher callback runs, so a publisher callback that immediately calls `pubads.refresh()` cannot observe a constituent call whose bookkeeping has not happened; +2. then invokes each caller's original handler in arrival order, passing a bid map **partitioned to that caller's ad-unit codes**, with throws isolated per handler. -**Return value:** Prebid 10's `requestBids` returns a promise, and the wrapper currently passes it through. Every held call therefore returns a promise that settles when the merged underlying `requestBids` promise settles, so callers awaiting the promise keep working; they observe the merged auction's completion rather than a per-call auction's. +**Promise and result semantics.** Prebid 10's `requestBids` returns a promise resolving `{bids, timedOut, auctionId}`. Every held call returns a promise that settles when the merged auction settles, with `bids` partitioned to the caller's codes and the shared `auctionId`. The shared auction id and merged event stream (one `auctionInit`/`auctionEnd` for N calls) are documented operator-visible changes; analytics consumers on the property see merged auctions. If the underlying dispatch throws synchronously, every pending promise rejects with that error and the queue resets. -### Lever B — GAM preconnect hint +### Lever B — GAM preconnect hint (opt-in) -The GPT integration's `head_inserts` emits, before its bootstrap scripts: +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -```html - -``` +When enabled, GPT `head_inserts` emits `` at head-start. -Unconditional: it is a pure hint with no behavioral effect, and every GPT-enabled property talks to this host. Removes DNS + TCP + TLS setup from the first ad request (and benefits the `pubads_impl` fetch). +**Corrections from review, reflected in scope and claims:** -### Lever C — first-auction prefetch at `DOMContentLoaded` (opt-in) +- This opens a connection to Google at head-start, **before consent resolution and on pages that may never issue an ad request**. That is a deliberate per-property privacy decision — hence config-gated, default off, for operators whose consent posture permits it. +- GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only help the **first direct ad request**. The claim is "may reduce" its connection setup; verification requires a HAR demonstrating connection reuse under the chosen credential mode. +- Credential mode matters: ad requests are cookie-credentialed, so the hint is emitted **without** `crossorigin` (credentialed preconnect). Browsers may partially perform or skip hints; this lever is best-effort by nature. -**Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. +### Lever C — earlier first auction (discovery first; design contingent) -**Mechanism today:** the `window.load` gate is `installSlimPrebidLoader` in the unified GPT bundle — the slim-Prebid module that runs refresh auctions is not even _loaded_ until `load` fires, and its first auction follows its post-load initialization. (The post-`load` + double-`rAF` defer from the React #418 fix belongs to the server-side-template `adInit` path and is not part of this flow.) +**Status: not implementation-ready.** Two review findings block a concrete design: the previously described lifecycle seam does not exist, and a raw `/auction` response cannot be "stashed and consumed" — the adapter maps responses back to Prebid bids via the requesting auction's Prebid-generated bid-request IDs. -**Behavior when enabled:** the unified GPT bundle — which is loaded from head-start — dispatches the first refresh auction's `/auction` request as soon as all of the following hold, without waiting for `window.load`: +**Phase 1 — discovery (required before any design commitment):** -- consent has resolved (the existing consent gate is unchanged), -- the GPT slots the auction would target have been observed, -- `DOMContentLoaded` has fired. +- Attribute the observed 3.3 s first `/auction` request precisely (it bypassed `pbjs.requestBids`; `slim_prebid_url` is unset on the measured property). Identify the triggering module, its gate (`load` listener, GPT event, publisher call), and what state it needs. +- Determine at what point the inputs a valid first auction needs are actually available: consent readiness (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), and publisher bidder params. -The response is stashed on `window.tsjs`. The slim-Prebid module, still loaded at the unchanged `window.load` point, consumes the stash during its existing initialization instead of issuing a fresh request. Only the network round trip moves earlier; script loading, targeting application, and the GPT refresh that triggers rendering keep their current post-load timing, so no render work moves into the hydration window. +**Phase 2 — design options to evaluate against discovery output:** -**Alternative considered and rejected:** loading the slim-Prebid script itself at `DOMContentLoaded`. Simpler, but it would also pull GPT refresh — and therefore ad rendering into publisher containers — earlier into the hydration window, the same territory as React #418 and the ad-container hydration gating tracked in #969. +1. **Transport-level cache behind the real `requestBids` lifecycle:** the adapter's transport layer may reuse an in-flight or completed `/auction` HTTP exchange when — and only when — the newly transformed request is byte-identical in its auction-relevant signature. Cache entries carry: exact transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, and one-shot consumed state. The response must expose a signal distinguishing consent-denied no-bid from legitimate no-bid (`/auction` currently returns HTTP 200 for both), and consent-denied responses are never cacheable. +2. **Classified full early auction:** run the real Prebid auction earlier and accept that scripts, events, consent modules, identity work, and native client-side bidders all move earlier. Honest but larger; interacts with hydration-window rendering (React #418, ad-container gating in #969) because earlier auctions pull GPT refresh earlier. -**Fallback:** if the stash is absent or its request has not resolved when slim-Prebid initializes, the module issues its own request exactly as today. A failed prefetch is discarded; no new error surface. +**Hard requirements for either option:** + +- **Consent readiness is a concrete API contract, not an assumption:** the trigger must consume the CMP signal (GPP `signalStatus: ready` / TCF `tcloaded`/`tcstring`, USP response) with a defined timeout and default action, and record the consent fingerprint used. The GPT bundle currently has no consent gate (it is documented as a future hook), so this gate must be built, not referenced. +- **No duplicate auctions:** an unresolved early request is **awaited** by the normal path (share the promise), never replaced by a second request. Cover pending→fallback→late-success, SPA navigation, and slot-destruction cases; a navigation or slot change invalidates the entry. +- **Billing integrity:** per the integrity section — early responses are targeting data only. ## Config-blob compatibility -Both new fields serialize only at non-default values, matching the repository's rollback discipline: a blob that never sets them is accepted by older binaries, and before rolling a binary back past this feature, an operator must clear the flags and re-push (the same procedure documented for prior `[integrations.prebid]` additions). +Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` **is** present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step is required. Testing includes a new-schema blob parsed by the legacy struct shape to lock this in. ## Testing -**Vitest (shim):** +**Vitest (shim), Lever A:** + +- two mergeable calls in the window → one underlying `requestBids`, union ad units, per-caller partitioned bid maps, handlers in arrival order; +- bookkeeping-before-callbacks: constituent-call registration observable before the first publisher callback runs; +- non-mergeable option keys flush then dispatch solo, preserving order; +- synthetic refresh auctions are never held (watchdog interplay covered with fake timers); +- calls containing client-side bidder entries are never held; +- duplicate/conflicting ad-unit codes flush before admission; payload-bound boundary flushes; +- unequal and absent timeouts: absolute-deadline adjustment, queue time counted, incompatible deadlines flush; +- throwing first handler does not block later handlers; synchronous dispatch failure rejects all pending promises and resets the queue; +- held-call promise resolves `{bids (partitioned), timedOut, auctionId (shared)}`; +- window `0` / absent config: existing suite passes untouched; +- callback reentrancy: a handler calling `requestBids` during the combined callback dispatches correctly. -- coalescing merges two mergeable calls in the window into one underlying `requestBids` with the union of ad units, and invokes both handlers in order; -- a call with extra option keys flushes the queue first and dispatches solo, preserving order; -- a throwing first handler does not prevent the second handler from running; -- window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); -- prefetch: a stashed response is consumed by slim-Prebid initialization without a second `/auction` request; an absent or unresolved stash falls back to the current request path; -- held `requestBids` calls return a promise that settles with the merged auction. +**Real-artifact coverage:** the external-bundle integration test asserts the wrapper's promise return against real Prebid (the current test discards the return value; the unit mock returns `undefined` and must not be the only coverage). **Rust:** -- config defaults (`request_bids_coalesce_ms = 0`, `prefetch_first_refresh_auction = false`) and the `0..=500` validation bound; -- injected-config serialization omits both fields at their defaults and includes them otherwise; -- GPT `head_inserts` contains the preconnect link. +- config defaults (`request_bids_coalesce_ms = 0`, `gam_preconnect = false`) and the `0..=250` bound; +- injected-config serialization omits `requestBidsCoalesceMs` at `0`; +- GPT `head_inserts` includes the preconnect link only when `gam_preconnect = true`, without `crossorigin`; +- new-schema → legacy-schema blob compatibility test. + +**Lever C tests are defined with its design after discovery** (consent denial/change, EID readiness, navigation/slot destruction, late completion, cache one-shot semantics). -**End-to-end verification:** re-run the local baseline harness (Viceroy proxying the production origin, pre-seeded consent, identical scroll script) with each flag enabled, comparing against the recorded baseline: time to first non-empty render, `/auction` request count, and burst dedup on the publisher's paired calls. +## Measurement methodology + +The three-run baseline motivates the work but does not gate it. Acceptance runs use: ≥10 runs per arm on the same machine and network, local Viceroy against the production origin with pre-seeded consent and an identical scripted scroll; medians compared, with a regression limit on first non-empty render (no worse than baseline median + 5%) and the target metric per lever (Lever A: `/auction` request count and burst dedup; Lever B: HAR-verified connection reuse on the first direct GAM request; Lever C: first-auction dispatch time). ## Rollout -1. Land binary with defaults off; the preconnect hint is the only immediate change. -2. Enable `request_bids_coalesce_ms` (initially 50 ms) and `prefetch_first_refresh_auction` on the autoblog tester property via config push. -3. Compare live tester metrics with the harness prediction; each flag reverts independently by config push, no binary rollback. +One lever at a time, each independently config-reversible: + +1. Land binary; all flags default off — zero behavior change. +2. Enable `request_bids_coalesce_ms` (50 ms) alone on the autoblog tester property. Guardrails: fill/revenue, bid rate, timeout rate, client-side bidder traffic (must be unchanged), duplicate-auction rate, beacons per rendered impression, per-slot render latency. +3. After A stabilizes, enable `gam_preconnect` alone; verify via HAR and consent-denied network activity monitoring (no pre-consent regressions beyond the documented connection). +4. Lever C follows its own spec revision after discovery. From 7bc12352002ced0569eb922bf62b37f0f0b0b38e Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 00:24:14 -0700 Subject: [PATCH 04/13] Address second review round on ad-latency spec Gate Lever A behind a burst-trace prerequisite (the baseline never recorded the burst calls' option keys or codes; same-code duplicate auctions would make the disjoint-code rule merge nothing), reframe bidder-QPS reduction as a hypothesis, document the shared-auction global-targeting caveat, make the deadline algorithm executable (bidderTimeout captured at enqueue, 50ms tolerance, monotonic flush at min(windowEnd, earliestDeadline - 100ms), 50ms dispatch floor so Prebid's timeout||bidderTimeout fallback never sees 0), add dispatch-time revalidation against in-place mutation with exact batch bounds, keep per-caller registration IDs so throw-rollback semantics survive merging, and generalize the flush-first rule to every ineligible arrival. Rewrite billing integrity per delivery path (the PUC bridge is not this scope's boundary; requestAds couples fetch to render; PBS server-side notices may bill off-render). Name tsjs.requestAds as Lever C's leading discovery hypothesis and add cookie-parity consent, server-owned equivalence inputs, and server-owned cache tokens. Add Lever B governance (net-log proof, GPC and denied-CMP cases, named approver, insertion-order test). Make measurement paired and thresholded, note the telemetry arm-label gap and read-once config rollback semantics, and sanitize property naming. --- ...-prebid-ad-latency-optimizations-design.md | 153 ++++++++++-------- 1 file changed, 89 insertions(+), 64 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 83d69a77f..3d8bf2af1 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,12 +1,12 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 -**Status:** Draft (returned from review; Lever C requires a discovery phase before implementation) -**Scope:** Client-side auction properties (server-side ad templates inactive) +**Date:** 2026-08-20 (revised 2026-08-21) +**Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) +**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements in this spec come from a pilot news property; identifying details are kept out of this document per repository policy, and sanitized measurement artifacts live outside the spec. ## Problem -On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the pilot property's origin (article page, consent resolved, reader-style scrolling) measured: | Milestone | Time | | -------------------------------------------------------------------- | ------ | @@ -17,120 +17,145 @@ On properties running the client-side auction path (`creative_opportunities.enab | Publisher's first `requestBids` | ~4.8 s | | First non-empty ad render | ~5.0 s | -Observed costs, each owned by a different lever below: +Observed costs, each owned by a lever below: -1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`.** The exact trigger of the observed 3.3 s `/auction` request is not yet attributed (it did not pass through `pbjs.requestBids`, and `gpt.slim_prebid_url` is not configured on the measured property). Lever C starts with a discovery task to attribute it precisely. -2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a server-load and bidder-QPS cost, not a first-render latency cost. Lever A is therefore a load/cost lever, not a latency lever. +1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`** and did not pass through `pbjs.requestBids`. The leading in-repo hypothesis is the `window.tsjs.requestAds` path, which builds its payload from the TSJS registry, POSTs `/auction` directly, and **immediately renders returned creatives** — fetch and render are coupled there. Lever C's discovery must confirm or refute this before any design commitment. +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, or payload sizes — the exact properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). 3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). ## Billing and impression integrity (applies to every lever) -Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see: +Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see. Because billing signals differ per delivery path, the requirement is stated per path: -- Prefetched or early-dispatched auctions are **targeting-only**: they produce bids, never renders. `nurl`/`burl` and any render-bridge activity remain tied to an actual GAM render of the slot, exactly as today. -- A prefetched bid that is never consumed expires without firing any beacon. -- Coalescing must not cause a caller's handler to render or fire beacons for another caller's ad units (see the partitioning rule in Lever A). -- Rollout guardrails (below) monitor duplicate-auction rate and beacon counts per rendered impression so any integrity regression is visible immediately. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer, and win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction; the invariant to preserve is that no lever triggers `pbjs` render or GPT refresh for units the publisher did not ask to render. +- **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. +- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). +- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope. It is listed here only to record that its beacon semantics are not the integrity boundary this spec relies on. +- A prefetched bid that is never consumed expires without firing any beacon; coalescing must not cause one caller's handler to render another caller's units (partitioning rule in Lever A, with the global-state caveat below). +- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness; no cross-path "beacons per impression" universal metric is claimed). ## Design -### Lever A — `requestBids` coalescing window (opt-in; load/cost reduction) +### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Objective:** reduce `/auction` request count and upstream bidder QPS for bursty publisher call patterns. Explicitly **not** claimed to reduce first-render latency; the hold can only add up to the window duration for the earliest caller, and rollout must verify render latency does not regress. +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and callback behavior. Implementation proceeds only if the observed calls satisfy every admission predicate below. If the burst turns out to be **same-code duplicate auctions** (which the shim supports today and the disjoint-code rule deliberately refuses to merge), Lever A as specified reduces nothing; the follow-up decision is then identical-request deduplication as a separate design, or dropping the lever. -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected into `window.__tsjs_prebid` as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**, not a claim: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. -**Admission (merge-safety) rules.** A call is held only when all of: +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. -- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array (any other key — `ortb2`, `labels`, `adUnitCodes`, `ttlBuffer`, `auctionId`, … — flushes the pending queue synchronously, then dispatches solo, preserving call order); -- it is **not** one of the shim's own synthetic refresh auctions (their GPT watchdog deadline starts when the wrapper returns; holding them races the watchdog and can discard valid bids); -- none of its ad units contains a bid entry for a configured client-side bidder (merging would merge those bidders' native auctions too, changing their request shape and analytics; if operators later want that, it is a separate, explicit decision); -- every held ad-unit `code` is non-empty and **disjoint** from the codes already pending (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types — a merged duplicate would produce a hybrid auction neither caller requested); -- the projected serialized payload of the merged request stays under a bound with safety margin (the endpoint rejects bodies over 256 KiB; bound pending calls, total ad units, and projected bytes — flush before admitting a call that would exceed any bound). +**Admission rules.** A call is held only when all of: -**Deadlines.** Each call's effective deadline is absolute: queue residence counts against it. A held call with timeout `T` arriving at `t0` must reach Prebid with an adjusted timeout of `T − (dispatch − t0)`. Calls merge only when their absolute deadlines are within a compatibility tolerance; an incompatible deadline flushes the queue. A call without a timeout uses the configured default for this computation. +- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; +- `timeout` is absent or a finite positive integer (zero, negative, `NaN`, `Infinity`, or non-number values dispatch solo unchanged); +- it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); +- no ad unit contains a bid entry for a configured client-side bidder; +- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected serialized payload of at most 192 KiB UTF-8 (64 KiB safety margin under the endpoint's 256 KiB limit). -**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the minimum adjusted deadline; and a combined `bidsBackHandler` that: +**Ineligible arrivals — one rule for all of them:** any call that fails any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing ever overtakes an earlier caller; event order is preserved. -1. first runs **all** Trusted Server bookkeeping for every constituent call (pending-publisher-bid registration, EID cookie sync) — before any publisher callback runs, so a publisher callback that immediately calls `pubads.refresh()` cannot observe a constituent call whose bookkeeping has not happened; -2. then invokes each caller's original handler in arrival order, passing a bid map **partitioned to that caller's ad-unit codes**, with throws isolated per handler. +**Deadlines.** -**Promise and result semantics.** Prebid 10's `requestBids` returns a promise resolving `{bids, timedOut, auctionId}`. Every held call returns a promise that settles when the merged auction settles, with `bids` partitioned to the caller's codes and the shared `auctionId`. The shared auction id and merged event stream (one `auctionInit`/`auctionEnd` for N calls) are documented operator-visible changes; analytics consumers on the property see merged auctions. If the underlying dispatch throws synchronously, every pending promise rejects with that error and the queue resets. +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`; a call without a timeout uses that captured value for deadline math. +- Each call's deadline is absolute: `arrival + effectiveTimeout`. Queue residence counts against it. +- Calls merge only when absolute deadlines agree within a **50 ms tolerance**; an incompatible deadline flushes the queue first. Later-arriving compatible callers accept the batch's earlier shared deadline and the shared `timedOut` result — documented behavior. +- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms safety margin)` and re-arms if a new caller tightens the earliest deadline. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms**; it is never `0` or negative (Prebid evaluates `timeout || bidderTimeout`, so `0` would silently restore the full global timeout). If the event loop wakes past a deadline (timer throttling, long tasks), dispatch immediately with the floor. + +**Dispatch-time revalidation.** The shim mutates publisher ad-unit objects in place, and publishers can mutate them further while a call is held; the adapter also builds the final payload (including then-current EIDs) only at dispatch. Every admission predicate — option keys, codes, bidders, bounds, projected serialized size — is therefore **re-checked at dispatch** against the live objects. A call that no longer qualifies is evicted from the batch and dispatched solo (current behavior); values that fail serialization (cycles, throwing getters/`toJSON`) are treated the same way. + +**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the floored shared deadline; and a combined `bidsBackHandler` that: + +1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, so the existing throw-rollback semantics are preserved per caller; +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`, with `bids` partitioned to that caller's codes and `timedOut`/`auctionId` shared. A throwing handler rolls back **only its own** registration (matching today's single-call behavior) and does not block later handlers. + +**Global-state caveat (documented behavior change).** Partitioned callbacks do not partition Prebid's global auction state: all merged bids belong to one auction, so a publisher callback that calls `pbjs.setTargetingForGPTAsync()` without codes applies targeting for every returned unit, and a bare `pubads.refresh()` can deliver another caller's units — all constituent calls' bookkeeping is registered before callbacks precisely so such a refresh is attributed as publisher delivery for every affected unit. Operators enabling the flag accept this shared-auction visibility; the test plan exercises unscoped targeting plus bare and mixed-slot refreshes from the first callback. + +**Promise and result semantics.** Every held call returns a promise settling with `{bids (partitioned), timedOut (shared), auctionId (shared)}`. A synchronous dispatch failure rejects all pending promises with that error; asynchronous rejection of the underlying promise fans out to all held promises; the queue resets in a `finally`. One `auctionInit`/`auctionEnd` event stream replaces N — a documented, operator-visible analytics change. ### Lever B — GAM preconnect hint (opt-in) **Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -When enabled, GPT `head_inserts` emits `` at head-start. +When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed, and the HTML preconnect algorithm keeps credentialed and anonymous connections distinct. Browsers may partially perform or skip hints; best-effort by nature. -**Corrections from review, reflected in scope and claims:** +**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -- This opens a connection to Google at head-start, **before consent resolution and on pages that may never issue an ad request**. That is a deliberate per-property privacy decision — hence config-gated, default off, for operators whose consent posture permits it. -- GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only help the **first direct ad request**. The claim is "may reduce" its connection setup; verification requires a HAR demonstrating connection reuse under the chosen credential mode. -- Credential mode matters: ad requests are cookie-credentialed, so the hint is emitted **without** `crossorigin` (credentialed preconnect). Browsers may partially perform or skip hints; this lever is best-effort by nature. +**Governance (required before any property enables it):** -### Lever C — earlier first auction (discovery first; design contingent) +- A per-property, per-jurisdiction approval that pre-consent DNS/TCP/TLS/SNI contact with Google is permitted, with a named approver — this hint fires at head-start on pages that may never issue an ad request. +- Verification uses browser **net logging**, not HAR alone: prove connection reuse by the first ad request under the credentialed mode, and prove **zero HTTP request bytes** are transmitted during speculation, including with GPC set and with the CMP unresolved or denied. +- Rollback trigger: any observed HTTP request on the speculative connection before the normal ad request disables the flag. -**Status: not implementation-ready.** Two review findings block a concrete design: the previously described lifecycle seam does not exist, and a raw `/auction` response cannot be "stashed and consumed" — the adapter maps responses back to Prebid bids via the requesting auction's Prebid-generated bid-request IDs. +**Config-surface note:** environment overrides cannot create absent GPT config leaves today (documented in the GPT guide); enabling per-property therefore goes through the pushed app-config blob, and the implementation updates `trusted-server.example.toml`, the configuration tables, and the GPT/Prebid guides. -**Phase 1 — discovery (required before any design commitment):** +### Lever C — earlier first auction (discovery first; design contingent) -- Attribute the observed 3.3 s first `/auction` request precisely (it bypassed `pbjs.requestBids`; `slim_prebid_url` is unset on the measured property). Identify the triggering module, its gate (`load` listener, GPT event, publisher call), and what state it needs. -- Determine at what point the inputs a valid first auction needs are actually available: consent readiness (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), and publisher bidder params. +**Status: not implementation-ready.** -**Phase 2 — design options to evaluate against discovery output:** +**Phase 1 — discovery (required):** -1. **Transport-level cache behind the real `requestBids` lifecycle:** the adapter's transport layer may reuse an in-flight or completed `/auction` HTTP exchange when — and only when — the newly transformed request is byte-identical in its auction-relevant signature. Cache entries carry: exact transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, and one-shot consumed state. The response must expose a signal distinguishing consent-denied no-bid from legitimate no-bid (`/auction` currently returns HTTP 200 for both), and consent-denied responses are never cacheable. -2. **Classified full early auction:** run the real Prebid auction earlier and accept that scripts, events, consent modules, identity work, and native client-side bidders all move earlier. Honest but larger; interacts with hydration-window rendering (React #418, ad-container gating in #969) because earlier auctions pull GPT refresh earlier. +- Attribute the 3.3 s `/auction` request precisely. **Named hypothesis:** the `tsjs.requestAds` path (posts `/auction` directly from the TSJS registry and immediately renders returned creatives). If confirmed, both Phase 2 options below are mis-scoped as written: moving `requestAds` earlier moves **rendering** earlier too, and its registry-built payload is not interchangeable with a later Prebid-adapter auction. The design must then first split fetch from render, and decide whether an independent non-Prebid auction on the page should be deduplicated or removed rather than accelerated. +- Identify the actual transport owner for any "adapter transport cache": today the Prebid adapter returns a request descriptor and **Prebid core owns the HTTP operation**; the repository-owned `sendAuction` belongs to the separate core API. A cache needs a named interception point in one of those owners. +- Determine when a valid first auction's inputs exist: consent (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), publisher bidder params. -**Hard requirements for either option:** +**Consent and equivalence model (server-owned inputs included):** -- **Consent readiness is a concrete API contract, not an assumption:** the trigger must consume the CMP signal (GPP `signalStatus: ready` / TCF `tcloaded`/`tcstring`, USP response) with a defined timeout and default action, and record the consent fingerprint used. The GPT bundle currently has no consent gate (it is documented as a future hook), so this gate must be built, not referenced. -- **No duplicate auctions:** an unresolved early request is **awaited** by the normal path (share the promise), never replaced by a second request. Cover pending→fallback→late-success, SPA navigation, and slot-destruction cases; a navigation or slot change invalidates the entry. -- **Billing integrity:** per the integrity section — early responses are targeting data only. +- CMP readiness in the browser is not the state the server consumes: the `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. A CMP can report granted while its cookie is absent or stale, so an early request could receive a consent-denied no-bid. Discovery must define how CMP readiness becomes the exact server-visible state — **cookie parity or an explicit validated consent envelope** — before any early dispatch. +- `/auction` returns HTTP 200 for both consent-denied and legitimate no-bid; a distinguishing response signal is required, and consent-denied responses are never reusable. +- Request equivalence is **not** body-byte equality: the server consumes EC identity, `ts-eids` fallback, KV-resolved EIDs, geo, IP, user agent, page identity, consent policy, and server configuration outside the body. Completed-response reuse is rejected unless the server provides a bounded, server-owned cache token covering those inputs. In-flight sharing ("no duplicate auctions": the normal path awaits the early request's promise) applies only while the equivalence snapshot — including consent and identity — is unchanged; a consent or identity change while pending invalidates/aborts the early request and runs a fresh normal one. +- Cache entries (if any) carry: transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, one-shot consumed state. +- Billing integrity per the section above; on the `requestAds` path, split fetch from render before any reuse. ## Config-blob compatibility -Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` **is** present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step is required. Testing includes a new-schema blob parsed by the legacy struct shape to lock this in. +Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step. Testing includes a new-schema blob parsed by the legacy struct shape. + +**Runtime note:** pages read the injected Prebid config once at load; a config rollback affects new navigations, not already-open pages. ## Testing **Vitest (shim), Lever A:** -- two mergeable calls in the window → one underlying `requestBids`, union ad units, per-caller partitioned bid maps, handlers in arrival order; -- bookkeeping-before-callbacks: constituent-call registration observable before the first publisher callback runs; -- non-mergeable option keys flush then dispatch solo, preserving order; -- synthetic refresh auctions are never held (watchdog interplay covered with fake timers); -- calls containing client-side bidder entries are never held; -- duplicate/conflicting ad-unit codes flush before admission; payload-bound boundary flushes; -- unequal and absent timeouts: absolute-deadline adjustment, queue time counted, incompatible deadlines flush; -- throwing first handler does not block later handlers; synchronous dispatch failure rejects all pending promises and resets the queue; -- held-call promise resolves `{bids (partitioned), timedOut, auctionId (shared)}`; -- window `0` / absent config: existing suite passes untouched; -- callback reentrancy: a handler calling `requestBids` during the combined callback dispatches correctly. +- two mergeable calls → one underlying `requestBids`, union units, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)` arguments; +- bookkeeping-before-callbacks with **per-caller registration IDs**: a throwing first handler rolls back only its own registration, later handlers still run, and the existing throw-rollback regression test still passes; +- every ineligible-arrival class (extra keys, synthetic refresh, client-side bidders, invalid timeout, duplicate/colliding codes, oversized payload, serialization failure) flushes the pending batch first, then dispatches solo — ordering asserted; +- global-state scenario: first callback performs unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()`; asserted against the documented shared-auction behavior; +- deadline math: timeout shorter than the window, zero/negative/`NaN` timeouts dispatch solo, `bidderTimeout` captured at enqueue survives a mid-hold config change, later caller with earlier deadline re-arms the scheduler, timer overshoot dispatches with the 50 ms floor (never `0`); +- dispatch-time revalidation: in-place mutation during the hold (code change, added client bidder, size growth) evicts to solo dispatch; cyclic/getter/`toJSON` failures evict to solo; +- promise semantics: async rejection fan-out, queue reset in `finally`, window `0`/absent config leaves the existing suite untouched; +- callback reentrancy: a handler calling `requestBids` during the combined callback. -**Real-artifact coverage:** the external-bundle integration test asserts the wrapper's promise return against real Prebid (the current test discards the return value; the unit mock returns `undefined` and must not be the only coverage). +**Real-artifact coverage (external bundle):** drive two real calls and prove one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, shared timeout/auction-id semantics, and rejection fan-out. (The unit mock returns `undefined` and the current artifact test discards the return value; neither may be the only promise coverage.) **Rust:** -- config defaults (`request_bids_coalesce_ms = 0`, `gam_preconnect = false`) and the `0..=250` bound; -- injected-config serialization omits `requestBidsCoalesceMs` at `0`; -- GPT `head_inserts` includes the preconnect link only when `gam_preconnect = true`, without `crossorigin`; +- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; +- `gam_preconnect = true` emits the link without `crossorigin` and **before** the GPT bootstrap inserts (transformed-HTML ordering test); `false` emits nothing; - new-schema → legacy-schema blob compatibility test. -**Lever C tests are defined with its design after discovery** (consent denial/change, EID readiness, navigation/slot destruction, late completion, cache one-shot semantics). +**Documentation checklist:** `trusted-server.example.toml`, configuration tables, Prebid and GPT integration guides, environment-overlay behavior note for GPT leaves. + +**Lever C tests are defined with its design after discovery.** ## Measurement methodology -The three-run baseline motivates the work but does not gate it. Acceptance runs use: ≥10 runs per arm on the same machine and network, local Viceroy against the production origin with pre-seeded consent and an identical scripted scroll; medians compared, with a regression limit on first non-empty render (no worse than baseline median + 5%) and the target metric per lever (Lever A: `/auction` request count and burst dedup; Lever B: HAR-verified connection reuse on the first direct GAM request; Lever C: first-auction dispatch time). +The three-run baseline motivates the work but does not gate it. Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **alternating paired control/treatment arms** on the same machine and network, explicit warm/cold connection conditions, ≥10 pairs per comparison, reporting p50/p95 and dispersion. + +Numeric gates: + +- **Lever A retains if:** ≥30% of observed eligible bursts merge (per the Phase 0 trace definition), total `/auction` requests per page drop ≥20%, and first-non-empty-render p95 regresses <5% vs. paired control. Otherwise the flag returns to `0`. +- **Lever B retains if:** net logs show connection reuse on the first direct GAM ad request in ≥70% of cold-start runs with zero pre-ad-request HTTP bytes; rollback on any integrity violation regardless of performance. +- Server telemetry currently labels all POSTs `auction_api` with no experiment-arm or navigation join key, so acceptance is **harness-based**; live guardrails during rollout are property-level trends (fill/revenue, bid rate, timeout rate, client-bidder traffic, duplicate-auction rate, per-slot render latency, consent-denied network activity) with a minimum 7-day hold per lever. ## Rollout -One lever at a time, each independently config-reversible: +One lever at a time, each independently config-reversible (new navigations only, per the runtime note): 1. Land binary; all flags default off — zero behavior change. -2. Enable `request_bids_coalesce_ms` (50 ms) alone on the autoblog tester property. Guardrails: fill/revenue, bid rate, timeout rate, client-side bidder traffic (must be unchanged), duplicate-auction rate, beacons per rendered impression, per-slot render latency. -3. After A stabilizes, enable `gam_preconnect` alone; verify via HAR and consent-denied network activity monitoring (no pre-consent regressions beyond the documented connection). +2. Lever A Phase 0 trace on the pilot property. If predicates hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrails; on failure return to `0` and **drain** (confirm zero merged auctions) before any further lever. +3. With A either retained-and-stable or fully drained to `0`: enable `gam_preconnect` alone under its governance contract. 4. Lever C follows its own spec revision after discovery. From e2c8de5e5bddb264c1bae73b752060d964fec2f2 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 00:50:24 -0700 Subject: [PATCH 05/13] Address third review round on ad-latency spec Resolve the integrity contradiction by consistently accepting cross-caller delivery within a merged auction (forbidden only for units no caller requested) and adding callback-discipline capture plus the pending-state/synthetic-auction net-effect estimate to Phase 0. Make the queue reentrancy-safe via atomic batch detachment before dispatch, reframe deadlines as a reduced auction-time budget with a solo-dispatch threshold instead of an absolute deadline, define order-preserving contiguous-segment eviction, snapshot request shape and ad-unit array order at enqueue, define the payload projection with a reserved adapter allowance and a 413 re-dispatch-solo-once fallback, and pass through undefined cancelled-auction values with handler throws reported by async rethrow rather than facade rejection. Scope Lever B approval to every jurisdiction the deployed config serves, pin the NetLog protocol, and narrow the invariant to no request HEADERS/DATA frames. Strengthen Lever C discovery with server-side single-flight preference, no completed-response reuse (the adapter re-stamps ttl 300), a typed direct-path result, and input-ownership tables, and remove the dangling Phase 2 reference. Add the adapter activation matrix (Fastly-only pilot), the guardrail source table with a new coalesced-count signal, drain over a 48h document lifetime, randomized paired methodology with power-based sample sizes, and a real latency-benefit gate for B. --- ...-prebid-ad-latency-optimizations-design.md | 143 ++++++++++-------- 1 file changed, 82 insertions(+), 61 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 3d8bf2af1..9bfc20f80 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,8 +1,8 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21) +**Date:** 2026-08-20 (revised 2026-08-21, round 3) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) -**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements in this spec come from a pilot news property; identifying details are kept out of this document per repository policy, and sanitized measurement artifacts live outside the spec. +**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). ## Problem @@ -20,61 +20,70 @@ On properties running the client-side auction path (`creative_opportunities.enab Observed costs, each owned by a lever below: 1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`** and did not pass through `pbjs.requestBids`. The leading in-repo hypothesis is the `window.tsjs.requestAds` path, which builds its payload from the TSJS registry, POSTs `/auction` directly, and **immediately renders returned creatives** — fetch and render are coupled there. Lever C's discovery must confirm or refute this before any design commitment. -2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, or payload sizes — the exact properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, payload sizes, or callback behavior — the properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). 3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). ## Billing and impression integrity (applies to every lever) -Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see. Because billing signals differ per delivery path, the requirement is stated per path: +Nothing in this design may create impression, win, or billing signals for ad units that **no caller requested to auction**. Signals differ per delivery path: -- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer, and win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction; the invariant to preserve is that no lever triggers `pbjs` render or GPT refresh for units the publisher did not ask to render. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller, and a publisher callback using unscoped targeting or a bare refresh may deliver units from a co-merged caller. What remains forbidden is delivery of units absent from every constituent call. - **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. - **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). -- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope. It is listed here only to record that its beacon semantics are not the integrity boundary this spec relies on. -- A prefetched bid that is never consumed expires without firing any beacon; coalescing must not cause one caller's handler to render another caller's units (partitioning rule in Lever A, with the global-state caveat below). -- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness; no cross-path "beacons per impression" universal metric is claimed). +- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. +- A prefetched bid that is never consumed expires without firing any beacon. +- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness); no cross-path "beacons per impression" universal metric is claimed. ## Design ### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and callback behavior. Implementation proceeds only if the observed calls satisfy every admission predicate below. If the burst turns out to be **same-code duplicate auctions** (which the shim supports today and the disjoint-code rule deliberately refuses to merge), Lever A as specified reduces nothing; the follow-up decision is then identical-request deduplication as a separate design, or dropping the lever. +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: -**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**, not a claim: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. +- If the burst is **same-code duplicate auctions** (supported today; the disjoint-code rule refuses to merge them), Lever A as specified reduces nothing — the follow-up is identical-request deduplication as a separate design, or dropping the lever. +- If callbacks perform **bare refreshes**, note the interaction: the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent common refresh pattern can then be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. + +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. + **Admission rules.** A call is held only when all of: - its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; -- `timeout` is absent or a finite positive integer (zero, negative, `NaN`, `Infinity`, or non-number values dispatch solo unchanged); +- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo (holding it would consume its budget; see Deadlines); - it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); - no ad unit contains a bid entry for a configured client-side bidder; - ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); -- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected serialized payload of at most 192 KiB UTF-8 (64 KiB safety margin under the endpoint's 256 KiB limit). +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected payload within the size budget below. + +**Payload size budget.** "Projected serialized payload" is defined as the UTF-8 byte length of `JSON.stringify` applied to the `/auction`-shaped body built from the snapshotted units at admission, re-computed at dispatch. Budget: projected units ≤ **160 KiB**, plus a **32 KiB reserved allowance** for adapter-added inputs (EIDs, context config — which the shim does not bound), leaving ≥ 64 KiB margin under the endpoint's 256 KiB limit. This is a best-effort bound, not a guarantee: if the server still rejects the merged body with 413, the coalescer **re-dispatches each constituent call solo, once** — no caller silently fails because of merging. Serialization failures (cycles, throwing getters/`toJSON`) at admission or dispatch evict that call to solo dispatch. -**Ineligible arrivals — one rule for all of them:** any call that fails any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing ever overtakes an earlier caller; event order is preserved. +**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid or sub-threshold timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. -**Deadlines.** +**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_ instead: -- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`; a call without a timeout uses that captured value for deadline math. -- Each call's deadline is absolute: `arrival + effectiveTimeout`. Queue residence counts against it. -- Calls merge only when absolute deadlines agree within a **50 ms tolerance**; an incompatible deadline flushes the queue first. Later-arriving compatible callers accept the batch's earlier shared deadline and the shared `timedOut` result — documented behavior. -- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms safety margin)` and re-arms if a new caller tightens the earliest deadline. -- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms**; it is never `0` or negative (Prebid evaluates `timeout || bidderTimeout`, so `0` would silently restore the full global timeout). If the event loop wakes past a deadline (timer throttling, long tasks), dispatch immediately with the floor. +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo (no deadline math is possible for them). +- Each call's nominal deadline is `arrival + effectiveTimeout`; queue residence counts against the budget. +- Compatibility: calls merge only while `maxDeadline − minDeadline ≤ 50 ms`; an incompatible arrival flushes first. Later-arriving compatible callers accept the batch's earlier shared budget and shared `timedOut` result — documented behavior. +- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms)` and re-arms if a new caller tightens the earliest deadline. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0` (Prebid's `timeout || bidderTimeout` would silently restore the global default). The floor is reachable only through timer overshoot (throttling, long tasks), because sub-threshold budgets were never admitted; overshoot means the auction runs up to ~50 ms past the nominal deadline, which is accepted and documented. -**Dispatch-time revalidation.** The shim mutates publisher ad-unit objects in place, and publishers can mutate them further while a call is held; the adapter also builds the final payload (including then-current EIDs) only at dispatch. Every admission predicate — option keys, codes, bidders, bounds, projected serialized size — is therefore **re-checked at dispatch** against the live objects. A call that no longer qualifies is evicted from the batch and dispatched solo (current behavior); values that fail serialization (cycles, throwing getters/`toJSON`) are treated the same way. +**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects (the shim mutates units in place and publishers can too). Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: for A(valid), B(now-invalid), C(valid), the dispatch order is merged-[A], solo-B, merged-[C] — never a reordering. (A single-segment queue with one invalid member yields exactly today's per-call behavior for that member.) -**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the floored shared deadline; and a combined `bidsBackHandler` that: +**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. Prebid invokes `bidsBackHandler` before resolving its public promise, so a constituent callback may re-enter `requestBids` and start a new batch while the first is settling — the detached-batch rule makes that safe, and the reentrancy test must prove the second batch dispatches and settles. -1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, so the existing throw-rollback semantics are preserved per caller; -2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`, with `bids` partitioned to that caller's codes and `timedOut`/`auctionId` shared. A throwing handler rolls back **only its own** registration (matching today's single-call behavior) and does not block later handlers. +**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order; the floored shared budget; and a combined `bidsBackHandler` that: -**Global-state caveat (documented behavior change).** Partitioned callbacks do not partition Prebid's global auction state: all merged bids belong to one auction, so a publisher callback that calls `pbjs.setTargetingForGPTAsync()` without codes applies targeting for every returned unit, and a bare `pubads.refresh()` can deliver another caller's units — all constituent calls' bookkeeping is registered before callbacks precisely so such a refresh is attributed as publisher delivery for every affected unit. Operators enabling the flag accept this shared-auction visibility; the test plan exercises unscoped targeting plus bare and mixed-slot refreshes from the first callback. +1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, preserving today's per-caller throw-rollback; +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; when Prebid supplies `undefined` (cancelled auction), `undefined` is passed through unaltered — as are `timedOut`/`auctionId`. A throwing handler rolls back only its own registration and does not block later handlers; the exception is reported by asynchronous rethrow (matching a lone call's observable behavior) and **does not reject any caller's facade promise**. -**Promise and result semantics.** Every held call returns a promise settling with `{bids (partitioned), timedOut (shared), auctionId (shared)}`. A synchronous dispatch failure rejects all pending promises with that error; asynchronous rejection of the underlying promise fans out to all held promises; the queue resets in a `finally`. One `auctionInit`/`auctionEnd` event stream replaces N — a documented, operator-visible analytics change. +**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: merged bids belong to one auction, so unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; the integrity section and acceptance tests treat cross-caller delivery within a merged auction as permitted. + +**Promise semantics.** Every held call returns a facade promise settling with the values described above. A synchronous dispatch failure rejects that segment's facade promises and resets only the detached segment. Prebid's public promise is resolve-only, so rejection fan-out is defensive unit/mock coverage — it cannot be proven against the real artifact without replacing the API under test. One `auctionInit`/`auctionEnd` event stream replaces N per merged segment — a documented, operator-visible analytics change. ### Lever B — GAM preconnect hint (opt-in) @@ -86,50 +95,48 @@ When enabled, GPT `head_inserts` emits `5% adverse move; owner: property operator) | existing, coarse | +| First-render / per-slot latency | sampled synthetic checks (scheduled harness runs against production), not RUM | synthetic | +| Consent-denied network activity (Lever B) | scheduled NetLog synthetic checks | synthetic | + +**Drain definition (Lever A):** after setting the flag to `0`, drain is complete when the `coalesced` signal reports zero merged auctions for a period covering the maximum expected open-document lifetime (48 h), since already-open documents keep their read-once config. ## Rollout -One lever at a time, each independently config-reversible (new navigations only, per the runtime note): +One lever at a time, Fastly pilot only, each independently config-reversible for new navigations: 1. Land binary; all flags default off — zero behavior change. -2. Lever A Phase 0 trace on the pilot property. If predicates hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrails; on failure return to `0` and **drain** (confirm zero merged auctions) before any further lever. -3. With A either retained-and-stable or fully drained to `0`: enable `gam_preconnect` alone under its governance contract. +2. Lever A Phase 0 trace on the pilot property (including callback-discipline capture). If predicates and net-benefit hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrail table; on failure return to `0` and confirm drain per the definition above. +3. With A either retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance contract, measured against the then-current A state. 4. Lever C follows its own spec revision after discovery. From 84bb7c10c9d4d393eb462fc33943e66ba5dfa85d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 11:14:01 -0700 Subject: [PATCH 06/13] Address fourth review round on ad-latency spec Replace the unimplementable post-response 413 replay with an authoritative size bound at the adapter buildRequests seam (constituent boundaries carried in; over-limit merged bodies split into multiple transport descriptors within one auction, preserving the single event stream). Replace the global async rethrow with a synchronous first-error throw after all handlers so Prebid's existing catch path handles it. Define the telemetry wire path as a top-level constituentCallCount body field through endpoint, observation context, event row, and a schema-first Tinybird column. Bound drain with a 24h injected-config lifetime and shim self-disable. Add the Fastly single-flight discovery exit criterion, extend the navigation-scoped reservation to both architectures, and specify the pending-only state machine with settled-result discard and cancellation honesty. Require a traffic-weighted browser matrix and durable approval artifact for Lever B with HTTP/1 write predicates and a versioned parser. Replace the circular eligible-burst denominator with candidate-burst coverage plus a 95 percent merge-success health gate, add a 30ms minimum worthwhile effect with named paired estimators, correct the Spin activation row, use a side-effect-free size estimator, add the segment-failure continuation test, and soften byte-for-byte wording. --- ...-prebid-ad-latency-optimizations-design.md | 155 ++++++++++-------- 1 file changed, 85 insertions(+), 70 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 9bfc20f80..a7c93d0b8 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21, round 3) +**Date:** 2026-08-20 (revised 2026-08-21, round 4) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -29,154 +29,169 @@ Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins Nothing in this design may create impression, win, or billing signals for ad units that **no caller requested to auction**. Signals differ per delivery path: -- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller, and a publisher callback using unscoped targeting or a bare refresh may deliver units from a co-merged caller. What remains forbidden is delivery of units absent from every constituent call. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller. What remains forbidden is delivery of units absent from every constituent call. - **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. -- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). -- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. +- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied. +- **The PUC render bridge** consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. - A prefetched bid that is never consumed expires without firing any beacon. -- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness); no cross-path "beacons per impression" universal metric is claimed. +- Guardrails use **per-path computable signals**; no cross-path "beacons per impression" universal metric is claimed. ## Design ### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: - If the burst is **same-code duplicate auctions** (supported today; the disjoint-code rule refuses to merge them), Lever A as specified reduces nothing — the follow-up is identical-request deduplication as a separate design, or dropping the lever. -- If callbacks perform **bare refreshes**, note the interaction: the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent common refresh pattern can then be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. +- If callbacks perform **bare refreshes**, the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent refresh can be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. -**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). + +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). The shim self-disables coalescing when `now > issuedAt + lifetime`, so long-lived documents quiesce on their own (see Drain). **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. **Admission rules.** A call is held only when all of: - its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; -- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo (holding it would consume its budget; see Deadlines); +- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo; - it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); - no ad unit contains a bid entry for a configured client-side bidder; -- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); -- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected payload within the size budget below. +- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending**; +- structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only. Units carrying accessors, custom `toJSON`, or otherwise unmeasurable values dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and an estimated unit payload ≤ 160 KiB UTF-8. -**Payload size budget.** "Projected serialized payload" is defined as the UTF-8 byte length of `JSON.stringify` applied to the `/auction`-shaped body built from the snapshotted units at admission, re-computed at dispatch. Budget: projected units ≤ **160 KiB**, plus a **32 KiB reserved allowance** for adapter-added inputs (EIDs, context config — which the shim does not bound), leaving ≥ 64 KiB margin under the endpoint's 256 KiB limit. This is a best-effort bound, not a guarantee: if the server still rejects the merged body with 413, the coalescer **re-dispatches each constituent call solo, once** — no caller silently fails because of merging. Serialization failures (cycles, throwing getters/`toJSON`) at admission or dispatch evict that call to solo dispatch. +**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — actually exists. The coalescer carries **constituent-call boundaries** into the merged auction, and `buildRequests` serializes the final body and, if it would exceed 192 KiB (64 KiB margin under the endpoint's 256 KiB limit), **splits along constituent boundaries and returns multiple transport descriptors** — Prebid dispatches each as a separate HTTP request within the _same_ auction. No oversized request is ever sent, no post-response replay exists, and the one-auction/one-event-stream contract holds. A server 413 is therefore not an expected outcome; if one occurs anyway it resolves as an ordinary no-bid auction exactly as an oversized solo call does today. -**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid or sub-threshold timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. +**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. -**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_ instead: +**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_: -- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo (no deadline math is possible for them). +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo. - Each call's nominal deadline is `arrival + effectiveTimeout`; queue residence counts against the budget. - Compatibility: calls merge only while `maxDeadline − minDeadline ≤ 50 ms`; an incompatible arrival flushes first. Later-arriving compatible callers accept the batch's earlier shared budget and shared `timedOut` result — documented behavior. - A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms)` and re-arms if a new caller tightens the earliest deadline. -- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0` (Prebid's `timeout || bidderTimeout` would silently restore the global default). The floor is reachable only through timer overshoot (throttling, long tasks), because sub-threshold budgets were never admitted; overshoot means the auction runs up to ~50 ms past the nominal deadline, which is accepted and documented. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0`. The floor is reachable only through timer overshoot, which means the auction runs up to ~50 ms past the nominal deadline — accepted and documented. -**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects (the shim mutates units in place and publishers can too). Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: for A(valid), B(now-invalid), C(valid), the dispatch order is merged-[A], solo-B, merged-[C] — never a reordering. (A single-segment queue with one invalid member yields exactly today's per-call behavior for that member.) +**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects. Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: A(valid), B(now-invalid), C(valid) → merged-[A], solo-B, merged-[C]. A synchronous dispatch failure of one segment rejects only that segment's facade promises; **later segments still dispatch in order and settle** (tested). -**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. Prebid invokes `bidsBackHandler` before resolving its public promise, so a constituent callback may re-enter `requestBids` and start a new batch while the first is settling — the detached-batch rule makes that safe, and the reentrancy test must prove the second batch dispatches and settles. +**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. A constituent callback may re-enter `requestBids` and start a new batch while the first settles — the detached-batch rule makes that safe, and the reentrancy test proves the second batch dispatches and settles. -**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order; the floored shared budget; and a combined `bidsBackHandler` that: +**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order (constituent boundaries attached for the adapter seam); the floored shared budget; and a combined `bidsBackHandler` that: 1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, preserving today's per-caller throw-rollback; -2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; when Prebid supplies `undefined` (cancelled auction), `undefined` is passed through unaltered — as are `timedOut`/`auctionId`. A throwing handler rolls back only its own registration and does not block later handlers; the exception is reported by asynchronous rethrow (matching a lone call's observable behavior) and **does not reject any caller's facade promise**. +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; `undefined` cancelled-auction values pass through unaltered. A throwing handler rolls back only its own registration and does not block later handlers; after **all** handlers have run, the **first captured error is thrown synchronously** so Prebid's existing catch-and-log path handles it — observably matching today's lone-call behavior. No global asynchronous rethrow. Facade promises still resolve. + +**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; acceptance tests treat cross-caller delivery within a merged auction as permitted. -**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: merged bids belong to one auction, so unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; the integrity section and acceptance tests treat cross-caller delivery within a merged auction as permitted. +**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. -**Promise semantics.** Every held call returns a facade promise settling with the values described above. A synchronous dispatch failure rejects that segment's facade promises and resets only the detached segment. Prebid's public promise is resolve-only, so rejection fan-out is defensive unit/mock coverage — it cannot be proven against the real artifact without replacing the API under test. One `auctionInit`/`auctionEnd` event stream replaces N per merged segment — a documented, operator-visible analytics change. +**Coalescing telemetry (required for rollout).** A dedicated, bounded, untrusted metadata field — `constituentCallCount` (integer, `2..=4`) — travels in the `/auction` request body as a **top-level sibling of** `config`, _not_ inside it (the `config` object is publisher auction context filtered by `allowed_context_keys`; unknown keys there are dropped). Semantics: absent = solo/legacy call; present only on merged dispatches; the count reaches `buildRequests` via the constituent boundaries the coalescer attaches, and split descriptors each carry their own segment's count. Server side: the endpoint parses and clamps the field, `AuctionObservationContext` and the auction event summary row carry it, and the Tinybird datasource gains the column — **schema deployed before the emitting binary**. Endpoint, telemetry, sink-serialization, and schema tests are part of the implementation, not optional. ### Lever B — GAM preconnect hint (opt-in) **Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed, and the HTML preconnect algorithm keeps credentialed and anonymous connections distinct. Browsers may partially perform or skip hints; best-effort by nature. +When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. + +**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. +**Browser coverage:** the flag emits to every browser, so verification cannot be Chromium-only. Governance requires a **traffic-weighted supported-browser matrix** (Chromium, WebKit/Safari, Firefox, material WebViews) with engine-appropriate low-level verification for each entry; engines that cannot be verified are documented as unverified in the approval. (Restricting emission per-engine would require request-scoped UA gating — out of scope, so the approval covers all engines the property serves.) **Governance (required before any property enables it):** -- The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; therefore the approval must cover **every jurisdiction served by the deployed configuration**, with a named approver. (Per-jurisdiction selectivity would require request-scoped consent/geo input to head insertion — out of scope here and stated as such.) -- **Verification protocol (NetLog):** pinned browser version, fresh profile per run, cache and socket pools cold, `--log-net-log` capture with sensitive data excluded, H2/H3/QUIC classified, and the speculative socket joined to the first GAM request via NetLog source IDs. Raw logs are redacted (no cookies/URLs beyond the GAM host) and retained only for the acceptance window. -- **Enforceable invariant:** no HTTP request HEADERS/DATA frames on the speculative connection before the normal GAM ad request. (Connection-level protocol frames — settings, pings — are inherent to preconnect and permitted.) Verified including GPC-set, CMP-unresolved, and CMP-denied cases. -- Rollback trigger: any observed request frames before the normal ad request disables the flag. +- The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; the approval must therefore cover **every jurisdiction served by the deployed configuration**. +- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. +- **Verification protocol:** pinned browser build per engine, fresh profile per run, cold cache and socket pools, capture mode stated explicitly, raw logs treated as sensitive (redacted to the GAM host, retained only for the acceptance window). For Chromium: NetLog with the speculative socket joined to the first ad request via source IDs, using a **versioned parser algorithm with fixtures and one controlled end-to-end test**; equivalent engine-appropriate tooling for others. Current audit tooling has no NetLog surface — building this capture path is part of the lever's implementation cost. +- **Enforceable invariant:** no HTTP request writes on the speculative connection before the normal first GAM ad request — defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. +- Rollback trigger: any observed request write before the normal ad request disables the flag — an immediate-stop conformance failure regardless of performance. ### Lever C — earlier first auction (discovery first; design contingent) -**Status: not implementation-ready.** No design option is selected in this revision; discovery below produces the inputs for a follow-up spec revision that will select one. +**Status: not implementation-ready.** No design option is selected; discovery produces the inputs for a follow-up spec revision. **Discovery contract:** -- **Attribute the 3.3 s `/auction` request.** Named hypothesis: `tsjs.requestAds` (posts `/auction` directly from the TSJS registry, invokes its callback, renders asynchronously). If confirmed, acceleration is mis-scoped until fetch is split from render, and the prior question becomes whether an independent non-Prebid auction on the page should be **deduplicated or removed** rather than accelerated. -- **Typed direct-path result.** `requestAds` returns `void` and `sendAuction` collapses network failure, parse failure, and legitimate emptiness into `[]`. A fetch/render split requires a private result type — e.g. `{outcome, bids, completedAt}` — distinguishing those cases while preserving the public callback lifecycle. -- **Transport ownership.** The Prebid adapter returns a request descriptor and **Prebid core owns that HTTP operation**; the repository-owned `sendAuction` belongs to the separate core API. Any cache/single-flight needs a named owner at one of those seams — and **server-side single-flight at the common auction boundary is the preferred shape**, because the browser cannot observe the HttpOnly EC identity, server-side KV/geo/identity-graph state, or configuration changes that determine auction equivalence. If a client-side mechanism is chosen anyway, it requires a short-lived, opaque, authenticated, navigation-bound server-issued token that reveals no stable identity and carries capability/configuration versioning. -- **No completed-response reuse initially.** The `/auction` response carries no bid lifetime and the adapter stamps a fresh `ttl: 300` at interpretation time, so reusing an old response silently renews its apparent lifetime. Reuse (if ever supported) requires completion time and per-bid expiry in the response, with TTL set to remaining lifetime. Until then, only **in-flight sharing** is on the table: the normal path awaits the early request's promise, valid only while the equivalence snapshot (consent, identity, navigation) is unchanged; any change invalidates/aborts and a fresh normal request runs. -- **Consent parity.** CMP readiness in the browser is not the state the server consumes: the `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. Discovery must define cookie parity or a validated consent envelope, and `/auction` needs a response signal distinguishing consent-denied from legitimate no-bid (both are HTTP 200 today); consent-denied results are never shareable. -- **Input ownership tables.** Producer/consumer/invalidation tables per path: TSJS registry generations, Prebid ad-unit generations, navigation generation, render targets. GPT slot targeting is listed as an auction input only if discovery proves it affects request bytes. -- **Billing integrity** per the section above; on the `requestAds` path, split fetch from render before any reuse. +- **Attribute the 3.3 s `/auction` request.** Named hypothesis: `tsjs.requestAds`. If confirmed, acceleration is mis-scoped until fetch is split from render, and the prior question becomes whether an independent non-Prebid auction should be **deduplicated or removed** rather than accelerated. +- **Typed direct-path result.** `requestAds` returns `void` and `sendAuction` collapses network failure, parse failure, and legitimate emptiness into `[]`. A fetch/render split requires a private result type — `{outcome, bids, completedAt}` — preserving the public callback lifecycle. +- **Transport ownership and the Fastly constraint.** The Prebid adapter returns a request descriptor (Prebid core owns that HTTP operation); the repository-owned `sendAuction` is the separate core API. Server-side single-flight at the common auction boundary is the preferred _shape_, **but it currently has no viable Fastly owner**: Fastly application state is rebuilt per request, and the platform abstraction exposes KV/cache/HTTP but no atomic pending-join primitive. **Discovery exit criterion:** prove a cross-instance, atomic, _pending-only_ join with zero retention after settlement on Fastly. If Fastly cannot supply that contract, server-side single-flight is off the table for the pilot and the candidate becomes authenticated client-side coordination or a different architecture. An ordinary persistent-cache lookup is not an acceptable substitute — it _is_ the completed-response reuse this spec forbids. +- **Navigation-scoped reservation (both architectures).** Client- or server-side, sharing requires: an opaque, authenticated, **single-navigation reservation**; exactly one intended early/normal pair per reservation; server revalidation of hidden inputs (HttpOnly EC identity, geo, server-resolved EIDs, headers, provider mode, configuration version) on the joining request; a canonical key derived from the fully normalized provider input plus relevant headers/settings — never a content-only or stable-identity key, which could join different documents or users with identical units and expose a one-shot bid across contexts. +- **Pending-only state machine.** Joining is allowed **only while Pending**. If no normal waiter attaches before settlement, the result is **discarded immediately**; a later normal call starts a fresh auction; the pending→settled attachment race is atomic and tested. Note the measured gap (early ~3.3 s, publisher ~4.8 s) makes settled-before-join the _likely_ case — the expected benefit of in-flight sharing is correspondingly modest and must be measured before further investment. +- **Cancellation honesty.** `sendAuction` has no `AbortSignal` and uses `keepalive`; client detach discards the local result but does not prove the server or upstream provider stopped. The design distinguishes **detach/result-discard** from **proven upstream cancellation**, adds generation guards against late targeting/render, and states whether a fresh auction may overlap an invalidated one. +- **Server-seam event semantics** (if a server join is ever built): share only provider/orchestrator execution after each waiter's inputs are normalized; serialize request-specific responses separately; emit one leader auction event plus a joined-waiter event; never replay the leader's correlation data. +- **No completed-response reuse.** The `/auction` response carries no bid lifetime and the adapter stamps a fresh `ttl: 300` at interpretation; reuse would silently renew lifetimes. If ever supported, the response must carry completion time and per-bid expiry, with TTL set to remaining lifetime. +- **Consent parity.** The `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. Discovery must define cookie parity or a validated consent envelope, and `/auction` needs a response signal distinguishing consent-denied from legitimate no-bid; consent-denied results are never shareable. +- **Input ownership tables** per path (TSJS registry generations, Prebid ad-unit generations, navigation generation, render targets); GPT slot targeting is an auction input only if discovery proves it affects request bytes. +- **Billing integrity** per the section above; split fetch from render before any reuse on the `requestAds` path. ## Config-blob compatibility -Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step. Testing includes a new-schema blob parsed by the legacy struct shape. +Integration settings are retained as raw JSON in the pushed blob, so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys. Compatibility tests carry the **non-default values** (`request_bids_coalesce_ms = 50`, `gam_preconnect = true`) through a full blob into the legacy struct shapes, plus present-leaf and absent-leaf environment-overlay tests for **both** fields. The public configuration guide's "unknown TOML keys fail" statement gains an explicit exception note for forward-compatible integration leaves. -**Adapter activation/rollback matrix.** Injected client config is read once per document, so _within_ an adapter a config change affects new navigations only. Across adapters, activation differs: Fastly instances are effectively per-request (config push suffices); Axum builds shared state at startup, and Cloudflare and Spin similarly hold startup state (config change requires restart/redeploy). **The pilot rollout is scoped to Fastly**; the other adapters inherit the flags but their activation path is restart-based and out of the pilot's scope. +**Adapter activation/rollback matrix.** Injected client config is read once per document, so within an adapter a config change affects new documents only — and "new navigation" is not "new document": client-path HTML can be browser-cached for 60 s. Across adapters: Fastly instances are effectively per-request (config push suffices). Axum builds shared state at startup (restart required). Cloudflare holds startup state (redeploy/restart). **Spin parses an embedded example config at build time — a restart cannot activate a changed flag; it requires a source-config change plus rebuild/redeploy (or future runtime config loading).** The pilot rollout is scoped to Fastly. ## Testing **Vitest (shim), Lever A:** -- two mergeable calls → one underlying `requestBids`, segment units in arrival order, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)` arguments; cancelled-auction `undefined` values passed through unpartitioned; -- bookkeeping-before-callbacks with per-caller registration IDs; a throwing first handler rolls back only its own registration, later handlers run, no facade promise rejects, the exception surfaces via async rethrow, and the existing throw-rollback regression test still passes; +- two mergeable calls → one underlying `requestBids`, segment units in arrival order, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)`; cancelled-auction `undefined` values pass through unpartitioned; +- bookkeeping-before-callbacks with per-caller registration IDs; a throwing first handler rolls back only its own registration, later handlers run, no facade promise rejects, and the first captured error is **synchronously** thrown after all handlers (asserted to land in Prebid's catch path, matching the existing lone-call regression); - every ineligible-arrival class flushes the pending batch first, then dispatches solo — ordering asserted; -- **order-preserving eviction:** middle-call mutation during the hold produces merged-[A], solo-B, merged-[C] dispatch and event order; -- shared-auction scenario: first callback performs unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()`; asserts the documented cross-caller delivery, pending-state consumption, and any resulting synthetic-auction classification (net `/auction` count asserted); -- **reentrancy:** a constituent callback enqueues a new batch while the first settles; the detached-batch rule is proven — the second batch dispatches and settles, and the first batch's cleanup does not clear it or its timer; -- deadline math: sub-threshold timeout dispatches solo; zero/negative/`NaN`/`Infinity` dispatch solo; missing/invalid captured `bidderTimeout` sends timeout-less calls solo; `bidderTimeout` captured at enqueue survives a mid-hold config change; a later caller with an earlier deadline re-arms the scheduler; timer overshoot dispatches with the 50 ms floor (never `0`), documented as budget overrun; -- snapshot semantics: post-enqueue request-key additions and ad-unit array push/splice do not alter the issued call; unit-object mutations are caught at revalidation; -- payload budget: boundary−1 / boundary / boundary+1 projections, multibyte (UTF-8) content, oversized EID allowance behavior, and the 413 → re-dispatch-solo-once fallback; -- async rejection fan-out and `finally`-scoped cleanup of the detached segment (defensive unit coverage; not provable against the real artifact); +- order-preserving eviction: merged-[A], solo-B, merged-[C]; **segment-failure isolation:** segment A's underlying dispatch throws synchronously, segments B and C still dispatch in order and settle; +- shared-auction scenario: unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()` from the first callback, asserting documented cross-caller delivery, pending-state consumption, synthetic-auction classification, and net `/auction` count; +- reentrancy: a constituent callback enqueues a new batch while the first settles; the detached-batch rule proven — second batch dispatches and settles un-clobbered; +- deadline math: sub-threshold, zero/negative/`NaN`/`Infinity`, missing/invalid captured `bidderTimeout`, mid-hold config change, earlier-deadline re-arm, overshoot floor (never `0`); +- snapshot semantics: post-enqueue request-key additions and array push/splice do not alter the issued call; unit-object mutations caught at revalidation; +- **side-effect-free estimator:** a stateful `toJSON`/getter is never invoked at admission (regression test); accessor-bearing units dispatch solo; +- **adapter split:** `buildRequests` with constituent boundaries splits an over-limit merged body into multiple descriptors along call boundaries (boundary−1/boundary/boundary+1, multibyte content, large EIDs), each carrying its segment's `constituentCallCount`; +- coalescing-config lifetime: coalescing self-disables past `issuedAt + lifetime`; - window `0` / absent config leaves the existing suite untouched. -**Real-artifact coverage (external bundle):** drive two real calls and prove one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, and shared timeout/auction-id semantics. (Rejection fan-out stays in unit coverage; the real public promise is resolve-only.) +**Real-artifact coverage (external bundle):** two real calls → one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, shared timeout/auction-id semantics; plus one real-artifact case where the merged body exceeds the limit and `buildRequests` returns multiple descriptors within one auction. **Rust:** -- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; -- `gam_preconnect = true` emits the link without `crossorigin` and **before** the GPT bootstrap inserts (transformed-HTML ordering test); `false` emits nothing; -- new-schema → legacy-schema blob compatibility test. +- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; injected coalescing config carries `issuedAt` + lifetime; +- `/auction` endpoint parses and clamps `constituentCallCount` (absent, 2..=4, out-of-range); `AuctionObservationContext` and the event summary row carry it; sink serialization includes it; a schema-ordering note ties the Tinybird column migration ahead of the emitting binary; +- `gam_preconnect = true` emits the link without `crossorigin` **before** the GPT bootstrap inserts; `false` emits nothing; +- new-schema → legacy-schema blob test with non-default values; present/absent env-leaf tests for both fields. -**Documentation checklist:** `trusted-server.example.toml`, configuration tables, Prebid and GPT integration guides, and environment-overlay leaf-creation behavior for **both** new fields (Prebid and GPT). +**Documentation checklist:** `trusted-server.example.toml`, configuration tables (including the unknown-keys exception note), Prebid and GPT integration guides, environment-overlay leaf behavior for both fields. **Lever C tests are defined with its design after discovery.** ## Measurement methodology -The three-run baseline motivates the work but does not gate it. Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering** on the same machine and network, explicit warm/cold connection conditions, and a sample size derived from pilot variance (a pilot batch of ≥10 pairs estimates variance; the acceptance batch is sized for 80% power on the stated effect, and is never smaller than 20 pairs). Decisions use **paired confidence bounds**, reporting p50/p95 with dispersion. +Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering**, explicit warm/cold connection conditions, and sample sizes derived from pilot variance with **quantile-specific power calculations** (a pilot batch of ≥10 pairs estimates variance; acceptance batches are powered at 80% for the stated effect at the stated quantile, never fewer than 20 pairs). Decisions use **named paired estimators**: median differences via the Hodges–Lehmann estimator with bootstrap 95% confidence intervals; non-inferiority stated per metric and quantile. -**Denominators, defined:** a _run_ is one full harness execution (fresh context); a _navigation_ is one document load within a run; an _eligible burst_ is a set of ≥2 `requestBids` calls within the configured window on one navigation that satisfy every admission predicate per the Phase 0 trace definition. +**Denominators:** a _run_ is one full harness execution (fresh context); a _navigation_ is one document load; a **candidate burst** is ≥2 `requestBids` calls within the configured window on one navigation (pre-admission); an **eligible burst** is a candidate burst whose calls pass every admission predicate. Numeric gates: -- **Lever A retains if:** ≥30% of eligible bursts merge, total `/auction` requests per navigation drop ≥20% (paired 95% CI excluding zero), **and** first-non-empty-render p95 shows non-inferiority within a 5% margin vs. paired control. Otherwise the flag returns to `0`. -- **Lever B retains if:** NetLog shows the speculative socket serving the first direct GAM ad request in ≥70% of cold-start pairs, the request-frame invariant holds in 100% of runs (any violation is an immediate rollback regardless of performance), **and** a real benefit gate passes: paired median first-GAM-request setup-time improvement with a 95% CI excluding zero, plus first-render and page-load non-inferiority (5% margin). Socket reuse alone does not retain the flag. -- **B is measured with identical Lever A state in both arms** (whatever A's disposition is at that point — the comparison is X vs. X+B, stated explicitly in the results). +- **Lever A retains if:** candidate-burst coverage is reported (share of candidate bursts that are eligible — this is the publisher-pattern fact); **merge success among eligible bursts is ≥95%** (implementation health — eligible bursts should merge absent runtime failure, and each failure is diagnosed); total `/auction` requests per navigation drop ≥20% (paired 95% CI excluding zero); and first-non-empty-render non-inferiority holds at p50 and p95 within a 5% margin. Otherwise the flag returns to `0`. +- **Lever B retains if:** the one-sided 95% **lower confidence bound** on cold-start speculative-socket reuse for the first GAM ad request exceeds 70%; the request-write invariant holds in 100% of runs (any violation = immediate rollback); and the benefit gate passes: paired median first-GAM-request setup-time improvement of at least the **minimum worthwhile effect of 30 ms** (Hodges–Lehmann, 95% CI excluding zero), with first-render (p50, p95) and `load`-event (p50) non-inferiority within 5%. Socket reuse alone does not retain the flag. +- **B is measured with identical Lever A state in both arms** — the comparison is X vs. X+B, stated in the results. -**Live guardrails and their sources.** Server telemetry today has no experiment arm, navigation ID, or coalesced-count signal, so live monitoring is explicitly split: +**Live guardrails and their sources:** -| Signal | Source | Type | -| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | -| Merged-auction count / drain state | new `coalesced` count field the shim adds to the `/auction` request `config` object; surfaced in server auction telemetry | new, required for Lever A rollout | -| `/auction` volume per property | existing server telemetry | existing | -| Fill/revenue, bid rate, timeout rate | ad-server / PBS reporting (property-level trend vs. 7-day pre-enable baseline, alert on >5% adverse move; owner: property operator) | existing, coarse | -| First-render / per-slot latency | sampled synthetic checks (scheduled harness runs against production), not RUM | synthetic | -| Consent-denied network activity (Lever B) | scheduled NetLog synthetic checks | synthetic | +| Signal | Source | Type / cadence | +| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- | +| Merged-auction count / drain state | `constituentCallCount` in server auction telemetry + Tinybird column | new; continuous | +| `/auction` volume per property | existing server telemetry | existing; continuous | +| Fill/revenue, bid rate, timeout rate | ad-server / PBS reporting vs. 7-day pre-enable baseline; alert on >5% adverse move; owner: property operator; reporting maturation delay 48 h | existing; daily | +| First-render / per-slot latency | scheduled synthetic harness runs against production | synthetic; daily during holds | +| Lever B socket reuse + setup time | scheduled engine-appropriate capture runs | synthetic; weekly during the B hold | +| Consent-denied network activity (Lever B) | scheduled capture runs (GPC / unresolved / denied) | synthetic; weekly during the B hold | -**Drain definition (Lever A):** after setting the flag to `0`, drain is complete when the `coalesced` signal reports zero merged auctions for a period covering the maximum expected open-document lifetime (48 h), since already-open documents keep their read-once config. +**Drain (Lever A):** the injected coalescing config expires after its 24 h lifetime and the shim self-disables, so drain is complete when the `constituentCallCount` signal shows zero merged auctions after `max(config lifetime, propagation bound, 60 s HTML freshness)` — a bounded guarantee, not an open-ended observation window. ## Rollout -One lever at a time, Fastly pilot only, each independently config-reversible for new navigations: +One lever at a time, Fastly pilot only, each independently config-reversible for newly generated documents: -1. Land binary; all flags default off — zero behavior change. +1. Land binary (including the telemetry field and Tinybird schema, deployed schema-first); all flags default off. 2. Lever A Phase 0 trace on the pilot property (including callback-discipline capture). If predicates and net-benefit hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrail table; on failure return to `0` and confirm drain per the definition above. -3. With A either retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance contract, measured against the then-current A state. +3. With A retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance artifact; **hold ≥7 days** with the weekly synthetic reuse/setup and consent-network checks; retain only if the live checks continue to meet the acceptance gates, roll back on any conformance failure immediately. 4. Lever C follows its own spec revision after discovery. From 401c6418cbf379f47af340800de7790b26e02a0e Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 22 Aug 2026 00:40:19 -0700 Subject: [PATCH 07/13] Address fifth review round on ad-latency spec Narrow the no-oversized-request guarantee to multi-constituent descriptors with an explicit singleton rule, key constituent boundaries by an internal auction ID read from the auction-scoped bidderRequest with a no-marker-leak test, and sanitize EIDs into plain data once per auction with unsanitizable EIDs making the body unsplittable. Replace constituentCallCount with a coalesced group/size/part object counted by distinct group, drop malformed values as absent, and name the auction_coalescing_daily rollup as the guardrail. Subordinate Lever C's pending-discard rule to one-consumer arbitration so the public requestAds render cannot be silently dropped, add the client-side real-artifact feasibility gate and the server-side side-effect-free prepare+fingerprint exit criterion, and require an outcome transition table. Block Lever B until every engine above 5 percent traffic is verified. Require the one-sided lower bound to exceed the 30ms effect, use a paired bootstrap quantile estimator for p95, define reuse denominators, split the causal harness from non-causal production guardrails with a fixed day-7 rule and daily batches, and make drain additive with a monotonic document lifetime and pipeline-liveness check. --- ...-prebid-ad-latency-optimizations-design.md | 46 +++++++++++-------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index a7c93d0b8..04d1858ec 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21, round 4) +**Date:** 2026-08-20 (revised 2026-08-22, round 5) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -63,7 +63,12 @@ Nothing in this design may create impression, win, or billing signals for ad uni - structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only. Units carrying accessors, custom `toJSON`, or otherwise unmeasurable values dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); - batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and an estimated unit payload ≤ 160 KiB UTF-8. -**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — actually exists. The coalescer carries **constituent-call boundaries** into the merged auction, and `buildRequests` serializes the final body and, if it would exceed 192 KiB (64 KiB margin under the endpoint's 256 KiB limit), **splits along constituent boundaries and returns multiple transport descriptors** — Prebid dispatches each as a separate HTTP request within the _same_ auction. No oversized request is ever sent, no post-response replay exists, and the one-auction/one-event-stream contract holds. A server 413 is therefore not an expected outcome; if one occurs anyway it resolves as an ordinary no-bid auction exactly as an oversized solo call does today. +**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: + +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used); the map entry is deleted at auction end. A test proves no boundary marker leaks into bidder params or the wire payload. +- **EID sanitization:** the adapter deep-copies collected EIDs into plain JSON data (own enumerable data properties only) **once per auction** before any serialization, so stateful getters/`toJSON` in publisher-supplied EID objects are never invoked repeatedly. EIDs that cannot be sanitized this way make the body **unsplittable** (single descriptor). +- **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. +- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single constituent whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. **Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. @@ -88,7 +93,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. -**Coalescing telemetry (required for rollout).** A dedicated, bounded, untrusted metadata field — `constituentCallCount` (integer, `2..=4`) — travels in the `/auction` request body as a **top-level sibling of** `config`, _not_ inside it (the `config` object is publisher auction context filtered by `allowed_context_keys`; unknown keys there are dropped). Semantics: absent = solo/legacy call; present only on merged dispatches; the count reaches `buildRequests` via the constituent boundaries the coalescer attaches, and split descriptors each carry their own segment's count. Server side: the endpoint parses and clamps the field, `AuctionObservationContext` and the auction event summary row carry it, and the Tinybird datasource gains the column — **schema deployed before the emitting binary**. Endpoint, telemetry, sink-serialization, and schema tests are part of the implementation, not optional. +**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` where `group` is an opaque per-logical-merge ID, `size` is the constituent count (`2..=4`), and `part`/`parts` describe split descriptors (`1/1` when unsplit). Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; malformed or out-of-range values are **dropped as absent, never clamped** into valid observations. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. ### Lever B — GAM preconnect hint (opt-in) @@ -98,7 +103,7 @@ When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. **Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -**Browser coverage (blocking gate):** the flag emits to every browser, and per-engine emission restriction would require request-scoped UA gating (out of scope). Therefore **enablement is blocked until every engine above a 5% traffic share on the property is verified** with engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging), each with defined event predicates, a per-engine sample floor (at least 20 cold-start runs), and per-engine acceptance rules. Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost — an unverified material engine blocks the flag; immaterial engines (below 5%) are documented as unverified in the approval. +**Browser coverage (blocking gates, two distinct scopes):** the flag emits to every browser, and per-engine emission restriction would require request-scoped UA gating (out of scope). Two gates with different scopes follow: + +- **Privacy/conformance gate — every emitted engine.** The no-request-write invariant is universal, so it must be verified on **every browser engine family that implements the preconnect hint** (Chromium/Blink, WebKit, Gecko — long-tail browsers embed these engines, which closes the enumeration), regardless of traffic share. An engine family that implements the hint and cannot be verified blocks enablement outright — a traffic threshold is never a privacy waiver. +- **Performance gate — material engines.** Reuse/benefit acceptance is required per engine **above a 5% traffic share** on the property; engines below 5% are documented as performance-unverified in the approval (their conformance is still required above). + +Each engine uses engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging) with defined event predicates, a per-engine sample floor (at least 20 cold-start runs), and per-engine acceptance rules. Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost. **Governance (required before any property enables it):** - The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; the approval must therefore cover **every jurisdiction served by the deployed configuration**. -- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. +- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. Its identifier and expiry are what the config's `gam_preconnect_approval_id`/`gam_preconnect_valid_until` carry, so runtime enforcement and the artifact cannot drift apart. - **Verification protocol:** pinned browser build per engine, fresh profile per run, cold cache and socket pools, capture mode stated explicitly, raw logs treated as sensitive (redacted to the GAM host, retained only for the acceptance window). For Chromium: NetLog with the speculative socket joined to the first ad request via source IDs, using a **versioned parser algorithm with fixtures and one controlled end-to-end test**; equivalent engine-appropriate tooling for others. Current audit tooling has no NetLog surface — building this capture path is part of the lever's implementation cost. -- **Enforceable invariant:** no HTTP request writes on the speculative connection before the normal first GAM ad request — defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. +- **Enforceable invariant:** no HTTP request writes on **any connection attributable to the hint** — not only the socket the first ad request later reuses — before the normal first GAM ad request; defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes, on every `securepubads.g.doubleclick.net` connection the browser opened speculatively. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. **Terminal observation horizon:** a run in which no GAM ad request ever occurs is not indeterminate — it observes all hint-attributed connections for the full scripted session plus 60 s of idle, and passes only if zero request writes occurred in that window. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. - Rollback trigger: any observed request write before the normal ad request disables the flag — an immediate-stop conformance failure regardless of performance. ### Lever C — earlier first auction (discovery first; design contingent) @@ -122,10 +128,10 @@ When enabled, GPT `head_inserts` emits ` Date: Sun, 23 Aug 2026 15:40:35 -0700 Subject: [PATCH 11/13] Address seventh review round on ad-latency spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Never evict live boundary entries — a full map dispatches new calls solo. Replace adapter-time EID sanitization with one raw-to-plain snapshot at collection reused by transport, split descriptors, bookkeeping, and cookie sync, with an explicit drop-and-count policy for non-serializable entries, retiring the per-constituent fallback. Make the drain bound hold end to end: config injection implies the 60s cache cap (cache-retaining response classes carry no config), document lifetime uses dual monotonic/wall-clock expiry with pageshow/visibilitychange resume guards, and propagation is observed across the served POP scope. Require a provider-free signed preflight token (input digest, navigation nonce, expiry; replay/mismatch/ navigation/expiry tests) for Lever C's client option, passive attach-rate measurement before any producer, admitted/rendered terminal outcomes, hard /auction and provider-execution non-regression gates, a keepalive 64 KiB transport gate, and blocking provider-log redaction. Turn Lever B's conformance gate into a finite product/OS/WebView matrix with an explicitly risk-accepted tail, bind emission to a signed approval manifest validated for hash, scope, and expiry, and record pre-consent DNS/TCP/TLS disclosure in the manifest. Use Wilson bounds for proportions with BCa reserved for paired continuous metrics, define no-render scoring, fix live control limits with minimum volumes and stale-data actions, constrain coalesced metadata to part <= parts <= size <= 4 with group consistency and a counted invalid-metadata signal, and surface the merged requestBids call visibility in Phase 0 and the real-artifact suite. --- ...-prebid-ad-latency-optimizations-design.md | 56 ++++++++++++------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index e8872c763..15bbb62d6 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-23, round 6) +**Date:** 2026-08-20 (revised 2026-08-23, round 7) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -49,7 +49,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). -**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp (diagnostic) and a maximum document lifetime (24 h). There is **one lifetime contract, and it is monotonic**: the shim latches config receipt at document start and tracks age with a `performance.now()`-based monotonic clock; it self-disables coalescing once document age exceeds the lifetime, so long-lived documents quiesce on their own regardless of wall-clock skew (see Drain, which uses the same contract). +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). There is **one lifetime contract — dual expiry with resume guards**: the shim latches config receipt and tracks a `performance.now()`-based monotonic document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`), self-disabling coalescing when either trips, re-checked on `pageshow`/`visibilitychange` so system suspension cannot extend the lifetime (see Drain, which relies on this exact contract). **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. @@ -65,11 +65,11 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: -- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop; the map is additionally bounded (oldest-entry eviction at 16 entries) so a cancellation path that reaches none of those hooks cannot grow it unboundedly. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. -- **EID sanitization:** the adapter deep-copies collected EIDs into plain JSON data (own enumerable data properties only) **once per auction** before any serialization; every descriptor serialization uses that sanitized copy, so stateful getters/`toJSON` in publisher-supplied EID objects run at most once per auction. This is not a new side effect: the current collector already retains `uid.ext` by reference and re-reads it at each serialization, so once-per-auction sanitization strictly reduces accessor invocations. -- **Sanitization-failure fallback:** if the sanitizing copy itself throws, the merged body is **never forced into one descriptor**. `buildRequests` falls back to **one descriptor per constituent** — byte-for-byte the bodies today's solo dispatches would have produced — so a failed sanitization can only reproduce solo behavior, not create a merged >256 KiB request out of individually valid calls. +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. +- **EID snapshot (one per auction, all consumers):** EID collection materializes the collected EIDs into plain JSON data (own enumerable data properties, one accessor read) **at collection time**, and that snapshot is the only EID representation any consumer sees — transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync all reuse it. Today the collector runs at three sites and retains `uid.ext` by reference, re-reading accessors at every serialization; the snapshot collapses those to one materialization per auction, strictly reducing accessor invocations. +- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. - **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. -- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case, in both the sanitized-split and per-constituent-fallback paths. +- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. **Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. @@ -92,13 +92,13 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; acceptance tests treat cross-caller delivery within a merged auction as permitted. -**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. +**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. **The visible `requestBids` calls themselves also change:** anything hooking or wrapping `pbjs.requestBids` (analytics adapters, publisher wrappers) observes fewer, later, merged invocations. Phase 0 must capture whether any such hook exists on the property, and the real-artifact suite documents the observable difference. -**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` with a **typed, bounded schema**: `group` is a client-generated UUIDv4 string (exactly 36 characters, format-validated); `size` is the constituent count (`2..=4`); `parts` is `1..=8` and `part` satisfies `1 <= part <= parts`. Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; any field failing its type, bound, or cross-field constraint drops the **whole object as absent, never clamped** into valid observations. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. +**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` with a **typed, bounded schema**: `group` is a client-generated UUIDv4 string (exactly 36 characters, format-validated); `size` is the constituent count; splits run along constituent boundaries, so the full cross-field constraint is `1 <= part <= parts <= size <= 4` (with `size >= 2`). Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; rows sharing a `group` must agree on `size`/`parts` (**group consistency**, validated in the rollup — a disagreeing group is counted invalid, not merged); any field failing its type, bound, or cross-field constraint drops the **whole object as absent, never clamped** — and every such drop increments a **`coalesced_invalid` counter** carried through the observation context, event row, and rollup, so malformed merged traffic cannot masquerade as solo and silently satisfy drain. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. ### Lever B — GAM preconnect hint (opt-in) -**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus two required-when-enabled siblings that make the governance artifact machine-enforced rather than merely recorded: `gam_preconnect_approval_id` (opaque string identifying the durable approval artifact) and `gam_preconnect_valid_until` (RFC 3339). Emission requires `gam_preconnect = true` **and** server time before `valid_until`; an expired or missing approval leaves the hint un-emitted and logs a warning at config load (automatic disable, no redeploy needed to stop emitting past expiry). +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus required-when-enabled siblings that bind emission to a **validated, signed approval manifest** rather than an arbitrary id and date: `gam_preconnect_approval_id` (identifier of the manifest), `gam_preconnect_approval_hash` (digest of the signed manifest content), and `gam_preconnect_valid_until` (RFC 3339, mirrored from the manifest). At config load the server validates the manifest reference: signature valid, hash matches, the manifest's recorded **configuration version and jurisdiction scope match the effective configuration**, and server time is before `valid_until`. Any failure leaves the hint un-emitted with a warning (automatic disable, no redeploy needed) — an approval that does not describe the running configuration proves nothing and must not enable emission. When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. @@ -106,7 +106,7 @@ When enabled, GPT `head_inserts` emits `48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. + **Live guardrails and their sources:** | Signal | Source | Type / cadence | @@ -204,7 +214,13 @@ Numeric gates: | Lever B socket reuse + setup time | scheduled engine-appropriate capture runs | synthetic; daily during the B hold | | Consent-denied network activity (Lever B) | scheduled capture runs (GPC / unresolved / denied) | synthetic; daily during the B hold | -**Drain (Lever A):** the shim tracks document age with a **latched monotonic clock** (`performance.now()`-based age, immune to wall-clock skew) and self-disables coalescing once the age exceeds the injected 24 h lifetime. Drain completion is an **additive** bound: **observed** config propagation + 60 s HTML freshness + 24 h document lifetime after the flag returns to `0`. Propagation is not asserted from a platform figure: after the disabling push, a **multi-POP probe** (requests routed through at least 3 distinct Fastly POPs, or the platform's config-version observability where available) must observe the new configuration before the propagation clock stops; ~5 minutes is the expectation, the observation is the bound. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. +**Drain (Lever A):** three requirements make the bound hold end to end. + +- **Injection implies the cache cap.** The 60 s freshness term is only valid for responses that actually carry it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject the coalescing client config**; any HTML response that injects either lever's config carries a browser-cache policy of at most 60 s. A test asserts the implication in both directions. +- **Document lifetime uses dual expiry with resume guards.** The shim tracks a latched monotonic (`performance.now()`-based) document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`); coalescing self-disables when **either** trips, re-checked on `pageshow`/`visibilitychange` so suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught at resume. Over-disabling is safe; only under-disabling breaks the bound. +- **Propagation is observed across the served scope.** After the disabling push, the effective **rendered** configuration must be observed across the POP scope actually serving the property (enumerated from recent traffic logs, or via platform config-version observability) — a fixed small probe count does not establish convergence for an eventually consistent config store; ~5 minutes is the expectation, the observation is the bound. + +Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the dual-expiry document lifetime after the flag returns to `0`. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges and no anomalous invalid-metadata counts while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. ## Rollout From 1aab56e32142f52d176c18aa1b4598863c38774d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sun, 23 Aug 2026 16:52:53 -0700 Subject: [PATCH 12/13] Address eighth review round on ad-latency spec Store constituent boundaries as code-to-membership sets partitioned in buildRequests, with unexpected or duplicate codes dispatching one unsplit descriptor and a post-handoff mutation test. Scope the EID snapshot to merged auctions with its own auction-keyed lifetime that outlives the boundary record, independent snapshots for overlapping reverse-settling auctions, an unchanged window-0 path, and a data-property walk that never invokes custom toJSON. Enforce the 24h lifetime fail-closed on every wrapped call and flush via monotonic age, elapsed wall time from receipt, and backward-jump disable, dropping the skew allowance. Make the Lever B approval an embedded JCS-canonical Ed25519-signed envelope with pinned rotating keys, a configuration digest computed at load, and per-emission expiry rechecks; stop emission at valid_until minus propagation and freshness so cached HTML cannot outlive approval. Extend the injection cache bound to CDN directives, Expires, and validators so an origin 304 cannot revive flag-carrying HTML. Bind Lever C's producer and joiner both to the preflight token, name the pending-join owner for single use, require pure fingerprints in both architectures, drop keepalive outright given the aggregate quota, widen the privacy audit to every provider and mediator including APS trace logs and client-visible debug bodies, and ban linkable stable hashes. Give live rollback rules degradation polarity with Bonferroni-split looks, restrict daily batches to feasible metrics with p95 on the accumulated pool, define the numeric no-render censoring value and the paired count-ratio estimator for Lever A's drop gate. Make the matrix cell product-OS-version-consent, preserve the full hint-transport graph before sanitization, add the navigation-keyed document span so the passive gate is computable, include coordination overhead in the load gate, and add map-capacity-at-flush, wrapper-order, invalid-metadata propagation, and group-consistency tests with a baseline-compared invalid signal that cannot hold drain open. --- ...-prebid-ad-latency-optimizations-design.md | 54 ++++++++++--------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 15bbb62d6..9f670b502 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-23, round 7) +**Date:** 2026-08-20 (revised 2026-08-23, round 8) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -49,7 +49,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). -**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). There is **one lifetime contract — dual expiry with resume guards**: the shim latches config receipt and tracks a `performance.now()`-based monotonic document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`), self-disabling coalescing when either trips, re-checked on `pageshow`/`visibilitychange` so system suspension cannot extend the lifetime (see Drain, which relies on this exact contract). +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp (diagnostic) and a maximum document lifetime (24 h). The contract enforces the 24 h bound, fail-closed, with no skew allowance: at config receipt the shim latches both the wall-clock time and the monotonic (`performance.now()`) reading, and **evaluates the expiry check on every wrapped `requestBids` call and every flush** (not only on lifecycle events). Coalescing self-disables when **any** of these holds: monotonic age ≥ 24 h; elapsed wall time since receipt ≥ 24 h; or the wall clock reads **earlier than the latched receipt time or the previous check** (a backward jump — fail closed, since elapsed time is no longer provable). A suspended monotonic clock is covered by the wall-time comparison at the next call/flush; a negatively skewed wall clock is covered by the backward-jump rule. The enforced lifetime is therefore **at most 24 h of provable elapsed time**, which is what Drain's additive bound uses. **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. @@ -65,9 +65,9 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: -- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. -- **EID snapshot (one per auction, all consumers):** EID collection materializes the collected EIDs into plain JSON data (own enumerable data properties, one accessor read) **at collection time**, and that snapshot is the only EID representation any consumer sees — transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync all reuse it. Today the collector runs at three sites and retains `uid.ext` by reference, re-reading accessors at every serialization; the snapshot collapses those to one materialization per auction, strictly reducing accessor invocations. -- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). Boundaries are stored as **code → constituent membership**, never numeric index ranges: between dispatch-time revalidation and `buildRequests`, Prebid hooks, ad-unit validation, and `beforeRequestBids` listeners can still remove or reorder units (or add a client bidder), so `buildRequests` partitions the units it actually receives by code membership. A received code that belongs to no constituent, or a duplicated code, marks the partition unreliable: the auction dispatches as a **single unsplit descriptor** with a diagnostic counter — never a mis-split. Post-handoff mutation (unit removal, reorder, and injected client bidder via a `beforeRequestBids` listener) is an explicit test. **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. +- **EID snapshot (merged auctions only; defined lifetime):** for a **merged** auction, EID collection materializes the collected EIDs into plain JSON data **at collection time** — walking own enumerable **data** properties with one accessor read and **never invoking custom `toJSON`** — and that snapshot is the only EID representation any of that auction's consumers see: transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync. The snapshot is keyed by the internal auction ID in its own map with its own lifetime — it **outlives the boundary record** (which is consumed at `buildRequests`) and is deleted only when the auction settles and its callback bookkeeping/cookie sync completes. Overlapping auctions settling in reverse order each hold an independent snapshot (tested). **Scope:** solo dispatches and the `window 0` path keep today's collection behavior byte-for-byte — snapshot and drop semantics apply only where merging is active, so the flag-off path is unchanged. +- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that merged auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. - **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. - **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. @@ -98,7 +98,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni ### Lever B — GAM preconnect hint (opt-in) -**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus required-when-enabled siblings that bind emission to a **validated, signed approval manifest** rather than an arbitrary id and date: `gam_preconnect_approval_id` (identifier of the manifest), `gam_preconnect_approval_hash` (digest of the signed manifest content), and `gam_preconnect_valid_until` (RFC 3339, mirrored from the manifest). At config load the server validates the manifest reference: signature valid, hash matches, the manifest's recorded **configuration version and jurisdiction scope match the effective configuration**, and server time is before `valid_until`. Any failure leaves the hint un-emitted with a warning (automatic disable, no redeploy needed) — an approval that does not describe the running configuration proves nothing and must not enable emission. +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus a required-when-enabled `[integrations.gpt.gam_preconnect_approval]` **embedded manifest envelope**, so validation needs no fetch and has no fetch-failure branch. The envelope carries the manifest content itself (id, configuration version, jurisdiction inventory, verified matrix summary, accepted tail, approver, `valid_until`), canonically encoded as **RFC 8785 (JCS) JSON**, with an Ed25519 signature over the canonical bytes. Trust and rotation: the verifying public keys ship in the binary's pinned trust set (two active keys during rotation overlap; a manifest verifies against either). At config load the server verifies: signature valid against a pinned key; the manifest's **configuration-version digest** (SHA-256 over the canonical encoding of the effective integration configuration, computed by the server at load) matches the digest the manifest records; and jurisdiction scope matches the deployed scope declaration. **Expiry is rechecked at every emission**, not only at load — GPT `head_inserts` builds synchronously from `Settings`, so the emission path compares current server time against `valid_until` per document; a long-lived adapter (Axum/Cloudflare hold startup state) therefore stops emitting at expiry without a reload. Any validation failure leaves the hint un-emitted with a warning — an approval that does not describe the running configuration proves nothing and must not enable emission. When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. @@ -109,13 +109,13 @@ When enabled, GPT `head_inserts` emits `48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. +**Live control limits (per signal, fixed before enablement):** every live guardrail carries a written tuple — estimator, fixed threshold, **rollback-bound polarity**, minimum daily volume, and missing/stale-data action — so a rollback decision is reproducible from the recorded rules alone. The polarity is the opposite of acceptance: acceptance proved benefit via a lower bound clearing a threshold, but copying that rule live would let mere daily-sample **uncertainty** trigger rollback. A live metric triggers rollback only on **evidence of degradation** — its one-sided 95% **upper** bound (for benefit metrics) falls below the minimum acceptable level, or a conformance invariant fails (those act immediately, no statistics involved). **Batching by metric feasibility:** daily 10-pair batches evaluate only median-level and proportion metrics; p95-gated metrics are evaluated once on the accumulated hold pool (≥70 pairs by day 7) at the day-7/day-9 read. **Repeated-look control:** the seven daily looks share a Bonferroni-split alpha (0.05/7 per look) so the hold's family-wise false-rollback rate stays at 5%. Fill/revenue alerts require a minimum volume of 1,000 impressions per property-day — a below-volume day is **inconclusive**, and inconclusive or missing/stale (>48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. **Live guardrails and their sources:** @@ -216,11 +219,12 @@ Numeric gates: **Drain (Lever A):** three requirements make the bound hold end to end. -- **Injection implies the cache cap.** The 60 s freshness term is only valid for responses that actually carry it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject the coalescing client config**; any HTML response that injects either lever's config carries a browser-cache policy of at most 60 s. A test asserts the implication in both directions. -- **Document lifetime uses dual expiry with resume guards.** The shim tracks a latched monotonic (`performance.now()`-based) document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`); coalescing self-disables when **either** trips, re-checked on `pageshow`/`visibilitychange` so suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught at resume. Over-disabling is safe; only under-disabling breaks the bound. +- **Injection implies a full-stack cache bound.** The 60 s freshness term is only valid when every cache layer honors it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject either lever's client config**. Any HTML response that does inject it carries: browser `Cache-Control: max-age=60` (or stricter), **capped or removed CDN-targeted directives** (`Surrogate-Control`, `Fastly-Surrogate-Control`, `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`), a removed `Expires`, and — critically — **no revivable validators**: `ETag`/`Last-Modified` are stripped (today conditional-header stripping applies only on ad-stack paths), because an origin `304` would otherwise revive previously transformed flag-carrying HTML past every `max-age`. A test asserts the implication in both directions across all named headers. +- **Emission stops before approval expiry can be outlived.** Lever B's per-emission expiry check uses `valid_until − (observed propagation + 60 s freshness)` as its effective cutoff, so no cached hint-carrying HTML can still be served when the approval lapses. +- **Document lifetime is enforced per call, fail-closed.** Per the lifetime contract above, expiry is evaluated on every wrapped call and flush against both the monotonic age and elapsed wall time since receipt, with backward wall-clock jumps disabling coalescing outright. Suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught by the wall-time comparison at the next call; over-disabling is safe, and no path extends the lifetime past 24 h of provable elapsed time. - **Propagation is observed across the served scope.** After the disabling push, the effective **rendered** configuration must be observed across the POP scope actually serving the property (enumerated from recent traffic logs, or via platform config-version observability) — a fixed small probe count does not establish convergence for an eventually consistent config store; ~5 minutes is the expectation, the observation is the bound. -Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the dual-expiry document lifetime after the flag returns to `0`. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges and no anomalous invalid-metadata counts while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. +Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the 24 h enforced document lifetime after the flag returns to `0` — and this **time bound is authoritative**. The telemetry check is confirmatory, not gating: because auction telemetry is best-effort and can drop rows, the `auction_coalescing_daily` rollup showing **zero logical merges while overall `/auction` row volume confirms pipeline liveness** corroborates drain; the invalid-metadata signal is compared against its **trailing pre-disable baseline** (a server-observed rate, not a value clients can set) and an anomaly **escalates to investigation without extending drain** — `coalesced` metadata is client-supplied, so a rule that let malformed traffic hold drain open indefinitely would hand an attacker a veto. Lever B rollback completion is the same observed propagation + 60 s HTML freshness. ## Rollout From 4fa73e7bdc6518eac9f381f07b2b0728ce389c4c Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 26 Aug 2026 21:01:27 -0700 Subject: [PATCH 13/13] Address ninth review round on ad-latency spec Make post-handoff client-bidder injection a Phase-0 blocking inventory item with explicitly accepted shared-auction semantics and a bidRequested-based runtime diagnostic, since buildRequests cannot see other bidders. Replace the unreliable-partition unsplit descriptor with per-constituent descriptors plus a residual descriptor so a mutated merged body can never become a whole-auction 413. Convert the 16-entry bound into a merged-auction ledger counted through settlement, holding boundary record and EID snapshot in one entry so slow auctions cannot accumulate unbounded state. Fix the approval digest to a versioned projection excluding the envelope itself, validated against an independently configured served-scope declaration, with signing test vectors. Define the single request-level injection predicate shared by head insertion, cache finalization, governance, and test denominators, moving GPC to a server-side link-absence test and keeping CMP-unresolved/denied as browser cells. Make the emission cutoff runtime-computable from a signed propagation budget and skew allowance, bound manifest lifetime at 90 days, and enumerate re-approval triggers. Return typed input_changed refusals before provider execution so a mismatched private producer never runs a consumerless auction. Scope fingerprinting to providers and require mediation to execute once inside the shared auction or make mediated properties ineligible. Define singleton segments as solo without coalesced metadata, extend the JSON-domain rule to the size estimator, and assert Prebid's public event stream in both wrapper orders. Move Lever B testing to material cells with continued-conformance triggers. Name the single-use owner per architecture (cross-instance primitive server-side, the document client-side) and derive the diagnostic digest from nonce plus digest. Define per-decision statistical families with Bonferroni-split alpha, one p95 read, symmetric no-render censoring, zero-baseline ratio exclusion, and navigation-clustered proportion inference. --- ...-prebid-ad-latency-optimizations-design.md | 38 ++++++++++--------- 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 9f670b502..b6835eb8e 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-23, round 8) +**Date:** 2026-08-20 (revised 2026-08-26, round 9) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -58,15 +58,15 @@ Nothing in this design may create impression, win, or billing signals for ad uni - its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; - `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo; - it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); -- no ad unit contains a bid entry for a configured client-side bidder; +- no ad unit contains a bid entry for a configured client-side bidder. Admission can only inspect what exists at call time: a Prebid hook or `beforeRequestBids` listener that injects a client bidder **after** dispatch-time revalidation is invisible to the adapter (its `buildRequests` sees only the Trusted Server bidder request) and cannot be detected or undone there. Three consequences: **Phase 0 must inventory bidder-mutating hooks/listeners on the property, and their presence blocks enablement**; if one nonetheless appears later, the injected bidder participates in the merged auction — this is **explicitly accepted shared-auction behavior** (the same cross-caller exposure the shared-auction section documents, now including a client bidder); and a runtime diagnostic detects it after the fact via the auction's `bidRequested` events (a non-admitted bidder observed in a merged auction increments a counter feeding the merge-health guardrail); - ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending**; -- structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only. Units carrying accessors, custom `toJSON`, or otherwise unmeasurable values dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); +- structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only, over the **JSON value domain**: units containing cyclic references, `BigInt`, function or symbol values, or non-finite numbers are unmeasurable and dispatch solo (the same domain rule the EID snapshot applies — non-JSON-domain EID entries are dropped under the non-serializable policy). Units carrying accessors or custom `toJSON` likewise dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); - batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and an estimated unit payload ≤ 160 KiB UTF-8. **Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: -- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). Boundaries are stored as **code → constituent membership**, never numeric index ranges: between dispatch-time revalidation and `buildRequests`, Prebid hooks, ad-unit validation, and `beforeRequestBids` listeners can still remove or reorder units (or add a client bidder), so `buildRequests` partitions the units it actually receives by code membership. A received code that belongs to no constituent, or a duplicated code, marks the partition unreliable: the auction dispatches as a **single unsplit descriptor** with a diagnostic counter — never a mis-split. Post-handoff mutation (unit removal, reorder, and injected client bidder via a `beforeRequestBids` listener) is an explicit test. **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. -- **EID snapshot (merged auctions only; defined lifetime):** for a **merged** auction, EID collection materializes the collected EIDs into plain JSON data **at collection time** — walking own enumerable **data** properties with one accessor read and **never invoking custom `toJSON`** — and that snapshot is the only EID representation any of that auction's consumers see: transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync. The snapshot is keyed by the internal auction ID in its own map with its own lifetime — it **outlives the boundary record** (which is consumed at `buildRequests`) and is deleted only when the auction settles and its callback bookkeeping/cookie sync completes. Overlapping auctions settling in reverse order each hold an independent snapshot (tested). **Scope:** solo dispatches and the `window 0` path keep today's collection behavior byte-for-byte — snapshot and drop semantics apply only where merging is active, so the flag-off path is unchanged. +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). Boundaries are stored as **code → constituent membership**, never numeric index ranges: between dispatch-time revalidation and `buildRequests`, Prebid hooks, ad-unit validation, and `beforeRequestBids` listeners can still remove or reorder units (or add a client bidder), so `buildRequests` partitions the units it actually receives by code membership. A received code that belongs to no constituent, or a duplicated code, marks the partition unreliable — and the fallback must never manufacture the oversized request splitting exists to prevent: the auction dispatches as **one descriptor per constituent** (cleanly attributed units, snapshot EIDs are plain data so this is always constructible) plus **one residual descriptor** carrying the unattributable units, each descriptor subject to the singleton oversized rule, with a diagnostic counter — never a forced merged descriptor and never a mis-split. Post-handoff mutation (unit removal, reorder, duplicate injection, and an injected client bidder via a `beforeRequestBids` listener) is an explicit test. **Capacity is a merged-auction ledger, counted through settlement:** each in-flight merged auction holds one ledger entry owning **all** of its module-scoped state — the boundary record (consumed at `buildRequests`, or on synchronous dispatch failure) **and** the EID snapshot (released at settlement, after callback bookkeeping and cookie sync complete) — with auction end/settlement as the backstop for the whole entry. The ledger is bounded at 16 **live entries measured until settlement**, so a slow auction whose boundary record was already consumed still counts against capacity while its snapshot lives; **live entries are never evicted** (evicting an active auction would silently lose split boundaries, snapshots, and telemetry). When the ledger is full, **new calls dispatch solo** (no new merges) until entries settle. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. +- **EID snapshot (merged auctions only; defined lifetime):** for a **merged** auction, EID collection materializes the collected EIDs into plain JSON data **at collection time** — walking own enumerable **data** properties with one accessor read and **never invoking custom `toJSON`** — and that snapshot is the only EID representation any of that auction's consumers see: transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync. The snapshot lives in the auction's ledger entry — it **outlives the boundary record** (which is consumed at `buildRequests`) and is released only when the auction settles and its callback bookkeeping/cookie sync completes, holding the entry's capacity slot the whole time. Overlapping auctions settling in reverse order each hold an independent snapshot (tested). **Scope:** solo dispatches and the `window 0` path keep today's collection behavior byte-for-byte — snapshot and drop semantics apply only where merging is active, so the flag-off path is unchanged. - **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that merged auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. - **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. - **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. @@ -81,7 +81,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni - A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms)` and re-arms if a new caller tightens the earliest deadline. - The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0`. The floor is reachable only through timer overshoot, which means the auction runs up to ~50 ms past the nominal deadline — accepted and documented. -**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects. Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: A(valid), B(now-invalid), C(valid) → merged-[A], solo-B, merged-[C]. A synchronous dispatch failure of one segment rejects only that segment's facade promises; **later segments still dispatch in order and settle** (tested). +**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects. Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: A(valid), B(now-invalid), C(valid) → merged-[A], solo-B, merged-[C]. A segment reduced to **one constituent is a solo dispatch** — it uses the ordinary solo path and carries **no `coalesced` metadata** (consistent with the schema's `size >= 2`). A synchronous dispatch failure of one segment rejects only that segment's facade promises; **later segments still dispatch in order and settle** (tested). **Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. A constituent callback may re-enter `requestBids` and start a new batch while the first settles — the detached-batch rule makes that safe, and the reentrancy test proves the second batch dispatches and settles. @@ -98,25 +98,27 @@ Nothing in this design may create impression, win, or billing signals for ad uni ### Lever B — GAM preconnect hint (opt-in) -**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus a required-when-enabled `[integrations.gpt.gam_preconnect_approval]` **embedded manifest envelope**, so validation needs no fetch and has no fetch-failure branch. The envelope carries the manifest content itself (id, configuration version, jurisdiction inventory, verified matrix summary, accepted tail, approver, `valid_until`), canonically encoded as **RFC 8785 (JCS) JSON**, with an Ed25519 signature over the canonical bytes. Trust and rotation: the verifying public keys ship in the binary's pinned trust set (two active keys during rotation overlap; a manifest verifies against either). At config load the server verifies: signature valid against a pinned key; the manifest's **configuration-version digest** (SHA-256 over the canonical encoding of the effective integration configuration, computed by the server at load) matches the digest the manifest records; and jurisdiction scope matches the deployed scope declaration. **Expiry is rechecked at every emission**, not only at load — GPT `head_inserts` builds synchronously from `Settings`, so the emission path compares current server time against `valid_until` per document; a long-lived adapter (Axum/Cloudflare hold startup state) therefore stops emitting at expiry without a reload. Any validation failure leaves the hint un-emitted with a warning — an approval that does not describe the running configuration proves nothing and must not enable emission. +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus a required-when-enabled `[integrations.gpt.gam_preconnect_approval]` **embedded manifest envelope**, so validation needs no fetch and has no fetch-failure branch. The envelope carries the manifest content itself (id, configuration version, jurisdiction inventory, verified matrix summary, accepted tail, approver, `valid_until`), canonically encoded as **RFC 8785 (JCS) JSON**, with an Ed25519 signature over the canonical bytes. Trust and rotation: the verifying public keys ship in the binary's pinned trust set (two active keys during rotation overlap; a manifest verifies against either). At config load the server verifies: signature valid against a pinned key; the manifest's **configuration digest** matches the server's recomputation over a **versioned digest projection** — the canonical (JCS) encoding of the effective GPT and relevant global configuration **with the `gam_preconnect_approval` envelope itself excluded** (the envelope cannot be inside the bytes it attests; the projection definition carries its own `projection_version`, recorded in the manifest, so binary and manifest cannot silently disagree about what was digested); and the manifest's jurisdiction inventory matches the **independently configured served-scope declaration** — an operator-maintained list outside the envelope (e.g., `[publisher] served_jurisdictions`) that is the runtime authority for what the deployment serves; the manifest attests it, it does not define it. The canonicalization-and-signature pipeline ships with **signing test vectors** (known-answer fixtures covering encoding, projection, digest, and signature). **Expiry is rechecked at every emission**, not only at load — GPT `head_inserts` builds synchronously from `Settings`, so the emission path compares current server time against `valid_until` per document; a long-lived adapter (Axum/Cloudflare hold startup state) therefore stops emitting at expiry without a reload. Any validation failure leaves the hint un-emitted with a warning — an approval that does not describe the running configuration proves nothing and must not enable emission. When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. **Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. +**One injection predicate (single source of truth).** Emission is decided by exactly one request-level predicate, and every consumer — HTML head insertion, cache finalization, the governance denominators, and the test denominators — evaluates the **same** predicate: GET HTML document navigation, not bot, not prefetch, **no server-visible consent denial** (`Sec-GPC` or a stored denial signal), approval valid, and before the emission cutoff. Predicate-false responses carry no hint and retain their existing cache policy; predicate-true responses carry the hint **and** the full flag-carrying cache bound (see Drain) — so the earlier tension between "emits to every browser" and the drain rule's injection restrictions is resolved: emission was never unconditional, it is exactly this predicate. Consent scenarios split accordingly: **server-visible** signals (GPC) are covered by a server-side test asserting the link is absent; **client-only** states (CMP-unresolved, CMP-denied — resolved in the browser after HTML was served) are covered by the browser conformance matrix with the link present, which is why those cells exist. + **Browser coverage (blocking gates, two distinct scopes):** the flag emits to every browser, and per-engine emission restriction would require request-scoped UA gating (out of scope). Two gates with different scopes follow: - **Privacy/conformance gate — a finite, named matrix, not a family claim.** One pinned build per engine family does not cover product, OS network-stack, WebView, fork, or version differences, so the gate is a **defined product/OS matrix**: for each engine family (Chromium/Blink, WebKit, Gecko), every product/OS combination above 1% property traffic share — **including Android WebView when the property serves it** — at pinned current stable versions. Each matrix cell must pass the invariant; a cell that cannot be verified blocks enablement — a traffic threshold is never a privacy waiver within the matrix. The **residual tail below the matrix floor is explicitly risk-accepted in the approval artifact** (named, with rationale: request-write behavior is an engine property and the tail embeds the verified engines), and any observed violation anywhere — matrix or tail — is an immediate rollback. - **Performance gate — material engines.** Reuse/benefit acceptance is required per engine **above a 5% traffic share** on the property; engines below 5% are documented as performance-unverified in the approval (their conformance is still required above). -The unit of verification is the **matrix cell — product × OS × pinned version × consent scenario** (consent-resolved, GPC-set, CMP-unresolved, CMP-denied) — not the engine: sample floors (at least 20 cold-start runs), event predicates, and acceptance rules are all defined and satisfied **per cell**, using engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging). Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost. +The unit of verification is the **matrix cell — product × OS × pinned version × consent scenario** (consent-resolved, CMP-unresolved, CMP-denied; server-visible GPC is a link-absence server test, not a browser cell, per the injection predicate) — not the engine: sample floors (at least 20 cold-start runs), event predicates, and acceptance rules are all defined and satisfied **per cell**, using engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging). Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost. **Governance (required before any property enables it):** - The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; the approval must therefore cover **every jurisdiction served by the deployed configuration**. - **The approval is a durable, signed manifest** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, the verified product/OS matrix and the explicitly risk-accepted tail, approver, date, expiry, a re-approval trigger when the served scope changes, and — explicitly — acceptance of the **pre-consent DNS/TCP/TLS disclosure inherent to preconnect**: the hint establishes a connection to the GAM host (disclosing the reader's IP address to it) before any consent signal resolves, an inherent property of the HTML preconnect algorithm that no request-write invariant removes. The signed manifest **is** the config's embedded `gam_preconnect_approval` envelope, so runtime enforcement and the artifact cannot drift apart. - **Verification protocol:** pinned browser build per engine, fresh profile per run, cold cache and socket pools, capture mode stated explicitly, raw logs treated as sensitive (retained only for the acceptance window). **Redaction must not destroy evidence:** the capture preserves the **complete hint-transport dependency graph** — every connection to the GAM host including coalesced HTTP/2/HTTP/3 connections, socket-to-request joins, and the initiator attribution proving the first ad request came from GPT — and sanitization to the GAM host happens only **after** the invariant is evaluated over that full graph, never before. For Chromium: NetLog with the speculative socket joined to the first ad request via source IDs, using a **versioned parser algorithm with fixtures and one controlled end-to-end test**; equivalent engine-appropriate tooling for others. Current audit tooling has no NetLog surface — building this capture path is part of the lever's implementation cost. -- **Enforceable invariant:** no HTTP request writes on **any connection attributable to the hint** — not only the socket the first ad request later reuses — before the normal first GAM ad request; defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes, on every `securepubads.g.doubleclick.net` connection the browser opened speculatively. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. **Terminal observation horizon:** a run in which no GAM ad request ever occurs is not indeterminate — it observes all hint-attributed connections for the full scripted session plus 60 s of idle, and passes only if zero request writes occurred in that window. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. +- **Enforceable invariant:** no HTTP request writes on **any connection attributable to the hint** — not only the socket the first ad request later reuses — before the normal first GAM ad request; defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes, on every `securepubads.g.doubleclick.net` connection the browser opened speculatively. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. **Terminal observation horizon:** a run in which no GAM ad request ever occurs is not indeterminate — it observes all hint-attributed connections for the full scripted session plus 60 s of idle, and passes only if zero request writes occurred in that window. Verified including CMP-unresolved and CMP-denied cases (server-visible GPC requests receive no link, asserted server-side); connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. - Rollback trigger: any observed request write before the normal ad request disables the flag — an immediate-stop conformance failure regardless of performance. ### Lever C — earlier first auction (discovery first; design contingent) @@ -130,12 +132,12 @@ The unit of verification is the **matrix cell — product × OS × pinned versio - **Transport constraint (`keepalive`).** `sendAuction` sends with `keepalive: true`, and the Fetch Standard's 64 KiB keepalive quota is **aggregate across unfinished keepalive requests, not per body** — two concurrent 40 KiB requests can fail even though each fits. A per-body size gate therefore cannot make `keepalive` safe; the private producer **drops `keepalive`** for its POSTs, with a test covering concurrent-request aggregation at the boundary. - **Provider/mediator privacy audit (blocking, all of them).** The redaction obligation is not Prebid-specific: the Prebid provider debug-logs the full normalized OpenRTB request, the APS provider `trace!`-logs its request body **and** exposes it as client-visible debug metadata when `debug` is on, and mediators see the same bytes. Before any fingerprint/preflight work ships, discovery delivers an **audit of every enabled provider and mediator across logs, error reports, telemetry, and client-visible debug output**, with blocking redaction tests. Retained diagnostics must be **bounded metadata or navigation-keyed values** — a stable content hash is linkable across navigations and is not an acceptable diagnostic form. - **Transport ownership and the Fastly constraint.** The Prebid adapter returns a request descriptor (Prebid core owns that HTTP operation); the repository-owned `sendAuction` is the separate core API. Server-side single-flight at the common auction boundary is the preferred _shape_, **but it currently has no viable Fastly owner**: Fastly application state is rebuilt per request, and the platform abstraction exposes KV/cache/HTTP but no atomic pending-join primitive. **Discovery exit criterion:** prove a cross-instance, atomic, _pending-only_ join with zero retention after settlement on Fastly. If Fastly cannot supply that contract, server-side single-flight is off the table for the pilot and the candidate becomes authenticated client-side coordination or a different architecture. An ordinary persistent-cache lookup is not an acceptable substitute — it _is_ the completed-response reuse this spec forbids. -- **Navigation-scoped reservation (both architectures).** Client- or server-side, sharing requires: an opaque, authenticated, **single-navigation reservation**; exactly one intended early/normal pair per reservation; server revalidation of hidden inputs (HttpOnly EC identity, geo, server-resolved EIDs, headers, provider mode, configuration version) on the joining request; a canonical key derived from the fully normalized provider input plus relevant headers/settings — never a content-only or stable-identity key, which could join different documents or users with identical units and expose a one-shot bid across contexts. **Key/fingerprint hygiene:** the key is a **versioned keyed digest** (server-keyed HMAC over the normalized input) combined with the navigation nonce; it lives only for the pending lifetime, and neither the digest input nor the raw digest ever appears in logs, URLs, or telemetry — telemetry may carry only a truncated, non-reversible diagnostic form. -- **Provider-free signed preflight (the only workable token issuance).** A digest computed during the early provider execution cannot reach the browser before that response settles — when pending-only joining is already closed — and a stateless instance cannot compare a joiner against inputs it never saw. The reservation token must therefore be issued by a **provider-free preflight**: it normalizes inputs, computes the digest, and immediately returns a **versioned, server-signed token carrying the input digest, navigation nonce, and expiry** — executing no provider call. **Both sides bind to the token:** the provider-executing early request presents it too, and the server revalidates the early request's inputs against the attested digest **before executing the provider** — a producer whose inputs mutated after preflight executes as an unshared fresh auction (its result is never joinable), so a joiner can never match a token whose producer ran different bytes. The joiner presents the same token; **any** instance validates the signature statelessly and compares its own revalidated inputs against the attested digest. **Single-use is enforced by the pending-join state owner** — the same atomic cross-instance pending-join primitive the Fastly exit criterion must prove; no other component claims that responsibility. The pure side-effect-free `prepare + fingerprint` capability is required in **both** architectures (it is what the preflight runs client-side-initiated, and what the server join compares). Discovery must deliver **replay, input-mismatch (joiner and producer sides), navigation-change, and expiry tests** for this protocol — without it, client coordination is rejected. +- **Navigation-scoped reservation (both architectures).** Client- or server-side, sharing requires: an opaque, authenticated, **single-navigation reservation**; exactly one intended early/normal pair per reservation; server revalidation of hidden inputs (HttpOnly EC identity, geo, server-resolved EIDs, headers, provider mode, configuration version) on the joining request; a canonical key derived from the fully normalized provider input plus relevant headers/settings — never a content-only or stable-identity key, which could join different documents or users with identical units and expose a one-shot bid across contexts. **Key/fingerprint hygiene:** the key is a **versioned keyed digest** (server-keyed HMAC over the normalized input) combined with the navigation nonce; it lives only for the pending lifetime, and neither the digest input nor the raw digest ever appears in logs, URLs, or telemetry — telemetry may carry only a diagnostic form derived from **both the navigation nonce and the digest** (a truncated HMAC keyed by the nonce), so a deterministic content digest cannot link the same inputs across navigations. +- **Provider-free signed preflight (the only workable token issuance).** A digest computed during the early provider execution cannot reach the browser before that response settles — when pending-only joining is already closed — and a stateless instance cannot compare a joiner against inputs it never saw. The reservation token must therefore be issued by a **provider-free preflight**: it normalizes inputs, computes the digest, and immediately returns a **versioned, server-signed token carrying the input digest, navigation nonce, and expiry** — executing no provider call. **Both sides bind to the token:** the provider-executing early request presents it too, and the server revalidates the early request's inputs against the attested digest **before executing the provider**. On mismatch the server returns a typed **`input_changed`/`ineligible` refusal without executing any provider** — a private producer's result could only ever reach a joiner, so a fresh unshared private execution would be a consumerless upstream auction (provider load and possible billing-relevant signals with no render path). Only a **public** request, which owns a consumer, may fall back to fresh execution; the private producer simply does not produce, and the later public call runs normally. A joiner therefore can never match a token whose producer ran different bytes — the mismatched producer never ran. The joiner presents the same token; **any** instance validates the signature statelessly and compares its own revalidated inputs against the attested digest. **Single-use has a named owner per architecture** — this is what dissolves the apparent contradiction between the Fastly constraint and the client fallback: in **server-side sharing**, the atomic cross-instance pending-join primitive (the Fastly exit criterion) owns it; in **client-side coordination**, the **document itself** is the atomic owner — single-threaded, navigation-scoped in-page state decides which consumer takes the result — and the server's role is limited to stateless token validation (signature, navigation nonce, expiry), which prevents cross-document and cross-navigation replay without any cross-instance state. Client coordination therefore does **not** require the primitive whose absence rules out server-side sharing. The pure side-effect-free `prepare + fingerprint` capability is required in **both** architectures (it is what the preflight runs client-side-initiated, and what the server join compares) — **scoped to providers**. Mediators cannot be fingerprinted at preflight: a mediator's request body depends on bidder responses and the remaining auction deadline, neither of which exists pre-execution. The attestation therefore covers **pre-provider inputs and the mediator's configuration identity only**, and mediation runs **exactly once inside the single shared execution**, over the shared response set, with its outcome delivered to the (single) consumer — or, if single-execution mediation cannot be designed for the configured mediator, **properties with a mediator configured are ineligible for sharing** and discovery says so explicitly. Discovery must deliver **replay, input-mismatch (joiner and producer sides), navigation-change, and expiry tests** for this protocol — without it, client coordination is rejected. - **Single-consumer sharing via a private fetch-only producer.** The public `requestAds` path renders every returned creative, and one bid must never reach two rendering consumers. Sharing is therefore restricted to a **private, fetch-only speculative producer**: its result carries no render obligation, and each response is **consumable exactly once** — the arbitration state machine hands the settled result to at most one consumer (the joining Prebid auction _or_ nothing), never to the public direct-path renderer and a second consumer. The public call keeps its normal consumer and its result is never shared. Removing or changing the public path's render remains permissible only as an explicitly approved breaking migration. - **Pending-only state machine (for a private speculative producer).** Joining is allowed **only while Pending**. If no waiter attaches before settlement, the private result is **discarded immediately**; a later call starts a fresh auction; the pending-to-settled attachment race is atomic and tested. The measured gap (early ~3.3 s, publisher ~4.8 s) makes settled-before-join the _likely_ case — expected benefit is modest and must be measured before further investment. A full **outcome transition table** is a discovery deliverable: for each of bid / no-bid / consent-denial / failure / timeout / invalidation / attachment-race, define the consumer, deadline, and terminal state. - **Client-side feasibility gate.** Prebid owns transport and request-scoped bid IDs, and the repository `sendAuction` returns flattened bids, losing raw response and outcome information. The invariant is **exactly one provider-executing `/auction` request** — not one HTTP request, which would contradict the server-revalidation requirement above. A lightweight, authenticated **join/claim request that executes no provider call** is permitted, and must be proven side-effect-free (no upstream contact, no billing-relevant state). Client-side coordination is admissible only after a **real-artifact feasibility proof**: one provider execution serving the single consumer while preserving bid-request IDs, APS admission, callbacks, promises, events, timeout semantics, targeting, and global bid state. If no supported seam exists, authenticated client coordination is also off the table — and Lever C may be infeasible in every architecture, which is an acceptable discovery outcome. -- **Server-side pure-plan seam.** Server request normalization currently inserts a fresh correlation UUID, and the provider contract exposes only a side-effecting `request_bids` — a fingerprint containing correlation randomness never matches, and invoking a provider to learn its exact bytes already contacts the upstream. **Discovery exit criterion:** a side-effect-free `prepare + fingerprint` capability for **every enabled provider and mediator**, or server-side sharing is off the table. +- **Server-side pure-plan seam.** Server request normalization currently inserts a fresh correlation UUID, and the provider contract exposes only a side-effecting `request_bids` — a fingerprint containing correlation randomness never matches, and invoking a provider to learn its exact bytes already contacts the upstream. **Discovery exit criterion:** a side-effect-free `prepare + fingerprint` capability for **every enabled provider**; mediators are excluded from fingerprinting (their inputs do not exist pre-execution) and are handled by the single-shared-execution rule in the reservation bullet — absent both, server-side sharing is off the table. - **Cancellation honesty.** `sendAuction` has no `AbortSignal` and uses `keepalive`; client detach discards the local result but does not prove the server or upstream provider stopped. The design distinguishes **detach/result-discard** from **proven upstream cancellation**, adds generation guards against late targeting/render, and states whether a fresh auction may overlap an invalidated one. - **Server-seam event semantics** (if a server join is ever built): share only provider/orchestrator execution after each waiter's inputs are normalized; serialize request-specific responses separately; emit one leader auction event plus a joined-waiter event; never replay the leader's correlation data. - **No completed-response reuse.** The `/auction` response carries no bid lifetime and the adapter stamps a fresh `ttl: 300` at interpretation; reuse would silently renew lifetimes. If ever supported, the response must carry completion time and per-bid expiry, with TTL set to remaining lifetime. @@ -164,8 +166,8 @@ Integration settings are retained as raw JSON in the pushed blob, so an explicit - **side-effect-free estimator:** a stateful `toJSON`/getter is never invoked at admission (regression test); accessor-bearing units dispatch solo; - **adapter split:** `buildRequests` reads boundaries via the internal auction ID from the auction-scoped `bidderRequest` and splits an over-limit merged body along call boundaries (boundary−1/boundary/boundary+1, multibyte content, large EIDs), each descriptor carrying correct `coalesced` group/part metadata; an **oversized single constituent** (large common EIDs) dispatches unsplit as today; **no boundary marker appears in bidder params or the wire payload**; a **stateful EID getter** is read exactly once per merged auction and a **custom `toJSON` is never invoked** (data-property walk), the transport body, split descriptors, and cookie sync all observe the same snapshot, overlapping reverse-settling auctions hold independent snapshots, the `window 0`/solo path collects EIDs exactly as today, and a **throwing EID materialization** drops that entry, increments the diagnostic counter, and the auction still dispatches; - coalescing-config lifetime: expiry is checked on every wrapped call and flush — a simulated suspend (frozen `performance.now()`, advanced `Date`) disables via the wall-time comparison, a forward-skewed `Date` alone disables no earlier than the monotonic bound, and a **backward `Date` jump disables immediately** (fail closed); -- boundary-map capacity: a queue admitted while the map is at 16 live entries flushes/dispatches solo without evicting any live entry, and merging resumes after cleanup releases entries; -- `requestBids` observability: exact timing and payload of the merged underlying call as seen by an external wrapper, in **both installation orders** (coalescer wraps the analytics wrapper; analytics wrapper wraps the coalescer); +- ledger capacity: new calls dispatch solo while 16 merged auctions are live, **including auctions whose boundary record is consumed but whose snapshot has not settled** (a slow-settling auction holds its slot); no live entry is ever evicted, and merging resumes after settlement releases entries; +- `requestBids` observability, in **both installation orders** (coalescer wraps the analytics wrapper; analytics wrapper wraps the coalescer): the external wrapper's observed call count, timing, and exact payload are asserted per order, **and Prebid's public event stream is asserted** — one `auctionInit`/`auctionEnd` pair per merged segment with the merged payload, at dispatch time, in both orders; - window `0` / absent config leaves the existing suite untouched. **Real-artifact coverage (external bundle):** two real calls → one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, shared timeout/auction-id semantics; a merged body exceeding the limit where `buildRequests` returns multiple descriptors within one auction, **including split-response aggregation** (each descriptor's response contributes its bids to the one auction result) and a **partial-413 case** (one descriptor rejected 413 while siblings succeed — the auction settles, surviving bids are delivered, no global error); a **throwing constituent callback** proving no global error surfaces and all facades resolve; and a merged dispatch where some constituents have **no handler**, proving bookkeeping still runs for them. @@ -179,7 +181,7 @@ Integration settings are retained as raw JSON in the pushed blob, so an explicit - `gam_preconnect = true` with a valid embedded approval envelope emits the link without `crossorigin` **before** the GPT bootstrap inserts; `false` emits nothing; enabled with a **missing envelope, signature failure, unknown signing key, configuration-digest or jurisdiction-scope mismatch, or expired `valid_until`** emits nothing and warns — each failure mode fixtured; **expiry is enforced per emission** (a valid-at-load manifest stops emitting once `valid_until` passes, tested with a stepped clock); - new-schema → legacy-schema blob test with non-default values; present/absent env-leaf tests for `request_bids_coalesce_ms`, `gam_preconnect`, and the `gam_preconnect_approval` envelope table. -**Lever B engine matrix:** for each engine family in the conformance gate, the matrix carries a versioned capture parser with fixtures, the privacy-invariant conformance run set (including the no-GAM-request horizon case), and one controlled end-to-end run — per engine, not Chromium-only. +**Lever B conformance matrix tests:** the unit of testing matches the unit of verification — **every material matrix cell** (product × OS × pinned version × consent scenario) carries a versioned capture parser with fixtures, the privacy-invariant conformance run set (including the no-GAM-request horizon case), and one controlled end-to-end run; nothing is tested only "per engine". **Continued conformance:** a pinned-version change in any cell, or a traffic-share crossing that makes a new cell material, re-runs that cell's set and triggers re-approval per the manifest rules before emission continues. **Documentation checklist:** `trusted-server.example.toml`, configuration tables (including the unknown-keys exception note and the preconnect approval keys), Prebid and GPT integration guides, environment-overlay leaf behavior for the new fields, and the **adapter activation/rollback matrix published in the operator documentation** (not only in this spec) so operators of non-Fastly adapters know a config change alone does not activate or roll back these flags. @@ -192,7 +194,7 @@ Integration settings are retained as raw JSON in the pushed blob, so an explicit ## Measurement methodology -Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering**, explicit warm/cold connection conditions, and sample sizes derived from pilot variance with **quantile-specific power calculations**: a pilot batch of ≥10 pairs seeds variance for **median** gates only; gates at p95 require a pilot batch of **≥30 pairs** (tail variance cannot be seeded from 10). Acceptance batches are powered at 80% for the stated effect at the stated quantile, never fewer than 20 pairs for median gates and never fewer than **50 pairs for p95 gates**. Decisions use **named paired estimators and a named interval method**: median differences via the Hodges–Lehmann estimator; quantile differences via the paired bootstrap; intervals for **continuous paired metrics** are **BCa bootstrap intervals with 10,000 resamples**; intervals for **binomial proportions** (socket reuse, merge success) use the **one-sided Wilson score bound** — BCa degenerates on all-success/all-failure samples and is not used for proportions; non-inferiority is stated per metric and quantile on a **ratio scale** (treatment/baseline at the stated quantile, one-sided 95% upper bound ≤ 1.05). **No-render handling:** a pair whose treatment arm produces no render within the harness window scores that arm at the **numeric measurement-window cap** (the harness observation window duration — a defined numeric censoring value usable in quantile and ratio estimators, conservative); pairs with no render in either arm are excluded but capped — more than 10% such pairs invalidates the batch. +Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering**, explicit warm/cold connection conditions, and sample sizes derived from pilot variance with **quantile-specific power calculations**: a pilot batch of ≥10 pairs seeds variance for **median** gates only; gates at p95 require a pilot batch of **≥30 pairs** (tail variance cannot be seeded from 10). Acceptance batches are powered at 80% for the stated effect at the stated quantile, never fewer than 20 pairs for median gates and never fewer than **50 pairs for p95 gates**. Decisions use **named paired estimators and a named interval method**: median differences via the Hodges–Lehmann estimator; quantile differences via the paired bootstrap; intervals for **continuous paired metrics** are **BCa bootstrap intervals with 10,000 resamples**; intervals for **binomial proportions** (socket reuse, merge success) use the **one-sided Wilson score bound** — BCa degenerates on all-success/all-failure samples and is not used for proportions; non-inferiority is stated per metric and quantile on a **ratio scale** (treatment/baseline at the stated quantile, one-sided 95% upper bound ≤ 1.05). **No-render handling (symmetric):** a run in **either arm** with no render inside the harness window scores that arm at the **numeric measurement-window cap** (a defined numeric censoring value usable in quantile and ratio estimators — applied identically to treatment and baseline, so censoring itself cannot bias the comparison); pairs with no render in **both** arms are excluded but capped — more than 10% such pairs invalidates the batch. **Zero-baseline ratios:** a pair whose baseline `/auction` count is zero has no defined ratio; such pairs are excluded from the count-ratio estimator, reported, and capped at 10% of the batch. **Clustering:** observations within one navigation are correlated, so burst-level proportions (merge success, eligibility) use a **cluster bootstrap resampling navigations**, not burst-level independence. **Denominators:** a _run_ is one full harness execution (fresh context); a _navigation_ is one document load; a **candidate burst** is ≥2 `requestBids` calls within the configured window on one navigation (pre-admission); an **eligible burst** is a candidate burst whose calls pass every admission predicate. @@ -204,7 +206,7 @@ Numeric gates: - **Causality boundary:** the pre-enable paired harness runs are the causal experiment and the **only basis for retention** — the acceptance gates above are evaluated on them, before enablement. The production hold checks are **non-causal guardrails** (a globally enabled property has no contemporaneous control arm): they can trigger rollback and nothing else. - **Hold decision rule:** synthetic checks run **daily** (batches of at least 10 pairs) as regression/conformance detectors, with conformance failures acting immediately at any point. "Hold completion" is not a retention decision: the hold **completes** when day 7 passes with no rollback trigger, and the reporting-based guardrails (fill/revenue, 48 h maturation delay) for the full day 1–7 window are evaluated once matured — **at day 9** — before the hold is declared clean. No sequential peeking on the synthetic batches; the day-9 reporting evaluation is the single final read. -**Live control limits (per signal, fixed before enablement):** every live guardrail carries a written tuple — estimator, fixed threshold, **rollback-bound polarity**, minimum daily volume, and missing/stale-data action — so a rollback decision is reproducible from the recorded rules alone. The polarity is the opposite of acceptance: acceptance proved benefit via a lower bound clearing a threshold, but copying that rule live would let mere daily-sample **uncertainty** trigger rollback. A live metric triggers rollback only on **evidence of degradation** — its one-sided 95% **upper** bound (for benefit metrics) falls below the minimum acceptable level, or a conformance invariant fails (those act immediately, no statistics involved). **Batching by metric feasibility:** daily 10-pair batches evaluate only median-level and proportion metrics; p95-gated metrics are evaluated once on the accumulated hold pool (≥70 pairs by day 7) at the day-7/day-9 read. **Repeated-look control:** the seven daily looks share a Bonferroni-split alpha (0.05/7 per look) so the hold's family-wise false-rollback rate stays at 5%. Fill/revenue alerts require a minimum volume of 1,000 impressions per property-day — a below-volume day is **inconclusive**, and inconclusive or missing/stale (>48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. +**Live control limits (per signal, fixed before enablement):** every live guardrail carries a written tuple — estimator, fixed threshold, **rollback-bound polarity**, minimum daily volume, and missing/stale-data action — so a rollback decision is reproducible from the recorded rules alone. The polarity is the opposite of acceptance: acceptance proved benefit via a lower bound clearing a threshold, but copying that rule live would let mere daily-sample **uncertainty** trigger rollback. A live metric triggers rollback only on **evidence of degradation** — its one-sided 95% **upper** bound (for benefit metrics) falls below the minimum acceptable level, or a conformance invariant fails (those act immediately, no statistics involved). **Batching by metric feasibility:** daily 10-pair batches evaluate only median-level and proportion metrics; p95-gated metrics get **exactly one read**, on the accumulated hold pool (≥70 pairs) at day 7. **The statistical family is defined per decision, and the confidence levels follow from it:** each decision family — A acceptance, A hold, B acceptance (including its per-material-cell performance gates), B hold — carries a total one-sided α of 0.05, **Bonferroni-split across every statistical test in the family** (looks × statistical signals, plus cells where applicable), so a family with _m_ tests runs each at α/m — i.e., per-test bounds are at `1 − 0.05/m` confidence, and the "95%" figures elsewhere describe the family-wise level, not each look. Conformance invariants (request-write, consent-network) are exact pass/fail checks and consume no alpha. Fill/revenue alerts require a minimum volume of 1,000 impressions per property-day — a below-volume day is **inconclusive**, and inconclusive or missing/stale (>48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. **Live guardrails and their sources:** @@ -220,7 +222,7 @@ Numeric gates: **Drain (Lever A):** three requirements make the bound hold end to end. - **Injection implies a full-stack cache bound.** The 60 s freshness term is only valid when every cache layer honors it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject either lever's client config**. Any HTML response that does inject it carries: browser `Cache-Control: max-age=60` (or stricter), **capped or removed CDN-targeted directives** (`Surrogate-Control`, `Fastly-Surrogate-Control`, `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`), a removed `Expires`, and — critically — **no revivable validators**: `ETag`/`Last-Modified` are stripped (today conditional-header stripping applies only on ad-stack paths), because an origin `304` would otherwise revive previously transformed flag-carrying HTML past every `max-age`. A test asserts the implication in both directions across all named headers. -- **Emission stops before approval expiry can be outlived.** Lever B's per-emission expiry check uses `valid_until − (observed propagation + 60 s freshness)` as its effective cutoff, so no cached hint-carrying HTML can still be served when the approval lapses. +- **Emission stops before approval expiry can be outlived — with a runtime-computable cutoff.** "Observed propagation" is only known after a disabling push, so the per-emission cutoff cannot depend on it. Instead the signed manifest carries a fixed **propagation budget** (15 min) and **clock-skew allowance** (5 min), and the effective cutoff is `valid_until − (propagation_budget + 60 s freshness + skew_allowance)` — every term a constant available at emission time. The post-rollback observed-propagation probe remains the rollback-completion measure; an observation ever exceeding the signed budget is a conformance failure requiring re-approval with a corrected budget. Manifest lifetime is bounded (`valid_until` at most 90 days from signing), and **re-approval triggers** are enumerated: browser/parser matrix change, trust-key rotation, configuration-digest change, and traffic-share crossings that move cells across the materiality thresholds. - **Document lifetime is enforced per call, fail-closed.** Per the lifetime contract above, expiry is evaluated on every wrapped call and flush against both the monotonic age and elapsed wall time since receipt, with backward wall-clock jumps disabling coalescing outright. Suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught by the wall-time comparison at the next call; over-disabling is safe, and no path extends the lifetime past 24 h of provable elapsed time. - **Propagation is observed across the served scope.** After the disabling push, the effective **rendered** configuration must be observed across the POP scope actually serving the property (enumerated from recent traffic logs, or via platform config-version observability) — a fixed small probe count does not establish convergence for an eventually consistent config store; ~5 minutes is the expectation, the observation is the bound.