Per-epoch collateral record: database, gossip serve, sampled sync, retention off by default

[MichaelTaylor3d]

Task

Track the per-epoch collateral record in the node's database, serve it over gossip, and sync it from
peers with the specified confidence.

Parent epic: https://github.com/DIG-Network/dig_ecosystem/issues/3173
Depends on: dig-mirror-collateral (the math) and the census in dig-mirror-coin.

What to store

Per epoch: the epoch number, census height, store count, distinct owner count, total locked, multiplier,
handicap, and the derived required_per_store — plus the inputs needed to re-derive it, so a record
can be audited rather than merely trusted.

Serve and sync

Any node may request any historical epoch's record over gossip.

Syncing uses the sample plan from the decision, which is chain-derived rather than assumed:
N is the count of distinct collateralised owner puzzle hashes, k(N) plateaus at 9 for a population
of 20 or more, and the adoption threshold is 6 of 9 — 99.7% under the stated assumption that at most
20% of the population is dishonest. Print the assumption beside the number wherever the confidence
is reported; a confidence figure without its assumption is not a claim.

Below the minimum population the sample is advisory only and the node derives from chain. At N=3 one
adversary is 33% and it is the assumption, not the statistics, that fails. Say so rather than reporting a
confidence the mathematics does not support.

Sample independently and at random. A sample drawn from one DHT bucket, or from a peer's referrals,
lets an adversary supply the population instead of a sample of it.

Disagreement is a signal, not noise. If the sample does not converge, say so and fall back to
deriving from chain — never adopt a plurality answer quietly.

An unrepresentative sample is now detectable: if the node hears from more distinct owners than the
chain says exist, that is a provable lie, not noise. Reject it.

The rule that makes all of this safe

A node's own computation always beats a peer's answer. Every record is recomputable from chain, so
the sample buys the ability to skip an expensive historical re-derivation — it never buys the right to
be wrong. If a synced record disagrees with what this node computes, this node's value wins and the
disagreement is reported.

Retention

A one-year truncation option, off by default. Document the consequence honestly: a truncated node
can no longer answer historical queries, and cannot re-derive recent epochs without re-scanning the
chain, because the recurrence needs its predecessors.

A dependency to plan around

dig-gossip is not published to crates.io (the index returns 404) and is consumed by git pin —
tracked as https://github.com/DIG-Network/dig_ecosystem/issues/3172. Either wait on that or put the
message types in a published crate. Decide deliberately and say which.

Method

TDD, hostile cases first: a peer serving a record that disagrees with local computation; a peer claiming
an epoch that does not exist yet; more distinct responders than the chain's owner count; a sample that
does not converge; a truncated node asked for a truncated epoch.

§2.4b, SPEC.md, and §2.6 as usual.

[MichaelTaylor3d]

Parent epic: https://github.com/DIG-Network/dig_ecosystem/issues/3173

The normative specification is the decider's comment on that epic — 16 sections, every constant, every integer expression, every rounding mode, plus a worked 10-epoch table that is the golden-vector seed. Its arithmetic was independently re-verified before these children were filed.

Two rules bind every child: no floating point anywhere in the consensus path (one ULP of divergence in the multiplier propagates into every later epoch and forks the network), and chain-only inputs — no DHT, no wall-clock, no local state, because every node has a different view of those.

[MichaelTaylor3d]

Binding amendment from the dig-mirror-collateral build — CALL the crate, do not re-derive the formula.

The sample plan (k(N), the agreement threshold, the assumed-dishonest bound) is a const fn in
dig-mirror-collateral. This node must call sync_sample_plan(N), never reimplement the arithmetic.

Two concrete reasons, both measured rather than theoretical:

1. The threshold has a floor of 1 that is structural inside the crate but not in the formula.
   threshold(k) = floor(2k/3) + 1 yields 0 at k = 0 under a literal two-thirds reading —
   "adopt anything on no evidence". The crate's form makes that unreachable and pins it with a test.
   A node that reapplies the formula from the spec text reintroduces the hole.
2. The threshold is 7 of 9, not 6. The decision's §9 formula and its annotation contradicted each
   other; the formula won and the documentation was corrected (see the SUPERSEDED banner at the top of
   the decision comment, and the ruling below it). A node re-deriving from the stale table would land on
   6 and be differently confident from the rest of the network — not a fork, since the sample is
   advisory, but a silent disagreement about how much evidence is enough.

Also note, for the reporting surface: max_assumed_dishonest uses floor division
(population / 5), so at a population of 20 it reports 4, not 5. The 20% assumption is therefore
enforced slightly stricter than stated wherever the population is not a multiple of five. That is
benign — it errs safe — but the figure this node reports to a user or a log should match what the crate
computes, not the round "20%" from the prose.

And the rule that outranks all of it: the sample is an optimisation. A node that disagrees with its
sample MUST prefer its own chain-derived computation. The threshold decides when re-derivation can be
skipped; it never decides what is true.

[MichaelTaylor3d]

Lane open on this ticket.

Branch loop/387-epoch-record, draft PR #398, head daab95d, branched from 2f0f148 (the #395 merge).

Building the four pieces: the per-epoch record database (I am the production writer for EpochRecordStore::put, which is test-only today), gossip serve, sampled sync via dig_mirror_collateral::sync_sample_plan, and retention off by default. Security shape is NC-12: a peer record is adopted only because it is recomputable, never because the peer is trusted.

[MichaelTaylor3d]

Ready for the gate. PR #398 at 36dbcab, still DRAFT.

All four pieces are in and exercised on a real node: the immutable per-epoch record store with its
first production writer (bring_up_collateral_records() at node start-up), the open peer serve
dig.getCollateralEpoch, the sampled-sync verification and adoption logic, and retention off by
default.

The security shape, in one line: a peer's record is adopted because it is recomputable, never
because the peer is trusted. A candidate is re-derived from a predecessor this node already holds
via EpochRecord::advance and every field must match — no formula is restated anywhere. What a
receiver cannot check is the census inputs against the chain, so that residue is bounded by the
sampling plan: unknown or too-small population never adopts, a plurality never adopts, more
responders than the chain-derived population refuses the whole sample, and an equivocating responder
gets no vote. The full statement is in the PR body and in SPEC.md §24.8–§24.11.

Running it found one defect a test had not: a two-epoch retention dropped epoch 1 and left the
store empty. Epoch 1 is the base case every verification walk terminates at, so it is now exempt
from every retention policy.

Two things I could not deliver, both stated plainly in the PR body rather than papered over:

1. dign collateral buffer still needs --roots, and D3's provenance marker stays.
   pairs_served_by_this_node is this node's OWN served set — a LOCAL fact. The record carries the
   NETWORK census, which control.rs:3283 and the contract both say is not a node count;
   multiplying it by the requirement would bill one operator for the whole network. dig-node has no
   notion of its own owner puzzle hash on this path, so any count it produced would be a resembling
   set — the exact error hostedStores.list made once, and invisible because both produce a
   plausible number. Needs its own ticket.
2. dign collateral requirement still answers unknown / not_censused for the current epoch.
   The only record writable without a chain source is epoch 1. Writing epoch n needs
   dig_mirror_coin::census, which takes a dig_chainsource_interface::ChainSource that dig-node
   neither implements nor depends on today. That is also why sampled sync's transport is not wired:
   without a chain source the population is unknown and adopt correctly returns Advisory on
   every call, so the loop could only ever refuse.

Suite: cargo test -p dig-node-service green across all targets, exit 0 (496 lib + every
integration target). fmt and clippy clean. Version 0.162.0 → 0.163.0.