Rate-limit grading fix (3.1.15) blocked from hosted grading — published_versions vs package_only_versions gap

[geneticallymodifiedfoodforthought]

Summary

A seller's badge is blocked because a rate-limit grading fix that shipped in 3.1.15 exists only in package_only_versions, not published_versions. Hosted grading (including manual "Test" runs) resolves a "3.1" target to a cached 3.1.13, which still hard-fails the same rate-limit step the fix was meant to resolve.

Details

• Seller: Elemental TV (Ilya Demidenko), original badge report in escalation Fix admin organization links and add clickable member names #518.
• Root cause identified: fix for replay_not_cached_rate_limit grades as failure when RATE_LIMITED never observed, contradicting storyboard's own not_applicable contract #6459 (rate-limit grading) shipped in 3.1.15.
• 3.1.15 is present in package_only_versions but not in published_versions.
• Hosted grading resolves "3.1" targets against published_versions, so it still uses cached 3.1.13 behavior — which hard-fails the rate-limit step even though the seller's agent is otherwise conformant.
• Net effect: seller cannot reach passing/badge status despite the fix already existing, because hosted grading can't see it.

Ask

1. Decide whether to promote 3.1.15 to published_versions so hosted grading picks up the fix.
2. If immediate promotion isn't viable, is there a seller opt-in path to grade against a package_only_versions release in the interim?

Impact

Blocks badge issuance for otherwise-conformant sellers whenever a fix lands in package_only_versions before being promoted — likely affects more than just this one case if the promotion cadence lags behind package releases.

Reported via

Escalation #542 (Ilya Demidenko), follow-up on escalation #518.

[bokelley]

Thanks for filing!

Triage

Classification: Bug (conformance failure — conformant seller blocked from badge)
Bucket(s): compliance-suite, admin/ops tools
Status: ready-for-human

What I found in the codebase:

static/compliance/published-versions.json confirms that 3.1.15 is in package_only_versions. However, 3.1.18 is already in published_versions — and since patches are cumulative, 3.1.18 should include the rate-limit fix from #6459.

This makes the issue's claim that grading resolves to 3.1.13 more puzzling: 3.1.18 is already published and should resolve as the latest stable 3.1.x. That gap points to a grading-service cache that hasn't been invalidated since 3.1.18 was added to the manifest — not only a missing promotion.

@bokelley, your call — two concrete questions:

1. Does 3.1.18 (already in published_versions) carry the fix from replay_not_cached_rate_limit grades as failure when RATE_LIMITED never observed, contradicting storyboard's own not_applicable contract #6459? If yes, the unblock is a hosted-grading cache flush, not a new promotion PR.
2. If the grading service pins a version independently of published_versions.json, what triggers a re-pin — and does Elemental TV's test run need a manual refresh now?

Systemic note: the jump from 3.1.13 → 3.1.18 in published_versions (skipping 3.1.15) suggests the promotion gate isn't consistently applied after each patch. Worth capturing the promotion criteria so future patches don't silently strand conformant sellers mid-certification.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01AL9L5Hfu1L69xWDcJi9cgc

---

Generated by Claude Code

[bokelley]

Added to the AdCP 3.2 RC verification-reliability gate tracked in #7061. This issue must be closed or explicitly removed from the published 3.2 RC verification profile with a documented limitation before the gate is complete.

[bokelley]

Hosted target-resolution rerun completed on 2026-08-30. Two manual production refreshes of Elemental TV (https://sales-agent.elementaltv.io/mcp) both selected requested target 3.1 and resolved the concrete bundle to 3.1.18 (run IDs 7a178d7e-bed8-4cd9-9775-599ed68671ab and 3187a358-52f5-40d4-bacc-0a8ca14c3604). The stale 3.1.13 / published_versions blocker described here is therefore no longer present in hosted grading.

The full runs did not reach the rate-limit scenario because the compliance runner rediscovered zero tools after the preceding capability probe found 10. That is a separate hosted discovery defect and is recorded on #6931; it does not change the target-resolution result this issue tracks.