diff --git a/Cargo.lock b/Cargo.lock index 0d6dfe4..2ee6656 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3067,7 +3067,7 @@ dependencies = [ [[package]] name = "dig-node-service" -version = "0.163.0" +version = "0.164.0" dependencies = [ "async-trait", "axum", diff --git a/Cargo.toml b/Cargo.toml index 2c96553..68de891 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -32,7 +32,7 @@ edition = "2021" # the ROOT manifest (`[workspace.package].version`), so it MUST be set here for a # release to fire (§3.6). The library crates (dig-node-core/dig-runtime/dig-wallet) # keep their own independent versions — only the released binary tracks the workspace version. -version = "0.163.0" +version = "0.164.0" # Release hardening, matching digstore: keep integer-overflow checks ON in release. # The node parses untrusted serialized input and does offset/length arithmetic over diff --git a/SPEC.md b/SPEC.md index d98d487..e69dca3 100644 --- a/SPEC.md +++ b/SPEC.md @@ -7611,4 +7611,139 @@ A malformed operator-supplied balance is REFUSED, never parsed as zero, which wo the node's own measurement otherwise render identically, which would make an operator's guess indistinguishable from a measurement in every figure derived from it. +* `collateral history [--epoch ]` — §24.8. Read from this node's own record store rather than + over a control call, so it answers on a node that is not running. Each line names the provenance, + because a bootstrap record, a censused one and one adopted from untrusted peers are three + different claims. An epoch never recorded MUST read as NOT RECORDED, distinctly from one recorded + and no longer readable. + Every verb offers `--json` beside the human output, with stable field names (§6.2). + +### 24.8. The per-epoch record store + +The node MUST persist one record per collateral epoch, in `collateral-epochs.jsonl` under its +machine-wide state directory, one JSON object per line. Each record carries the consensus +`EpochRecord` — the census inputs (advertised stores, collateralised owners, multiplier, handicap), +the derived `required_per_store_dig_base_units`, and the `protocol_version` that computed it — plus +two node-local fields: + +* `census_height` — the block height the census behind the record was taken at, or `null` at epoch 1, + which is derived from nothing and was taken at no height. `null` MUST mean "no census was taken", + never "the height was lost". +* `provenance` — `bootstrap`, `censused`, or `adopted_from_peers` with the counts that agreed and + answered. The three are different claims about what this node verified and MUST NOT be rendered + identically. + +The census height MUST NOT enter the arithmetic. Two nodes reading the same chain at the same height +derive the same record whether or not either records the height; it is carried so a disputed census +names the block it can be re-run against. + +**Historical records are permanent and immutable.** A record that DIFFERS from one already held for +that epoch MUST be refused and the held one kept. A node that let the newest writer win could be +walked off the network's history one epoch at a time by whoever spoke last, and every figure below +it — including the amount of $DIG the operator posts — would follow. An identical record offered +with stronger provenance MAY be recorded, because the consensus figures do not move; the reverse +MUST NOT be, because evidence does not weaken on re-offer. + +A node MUST record the epoch-1 record at start-up if it holds none. It depends on nothing, so every +node can produce it, and it is the base case that makes the recurrence well founded. + +**A record whose `protocol_version` exceeds what the build implements MUST NOT be served as +authoritative** — not by `control.collateral.requirement`, not by `dig.getCollateralEpoch`, and not +into a verification. Every field of such a record parses, so nothing downstream would question its +figures. + +### 24.9. Serving an epoch to a peer + +`dig.getCollateralEpoch` is an OPEN node method taking `{ epoch }` and returning `{ record }` or +`{ record: null, reason }`. It is unauthenticated because an epoch record carries nothing secret and +a caller verifies what it receives by re-derivation whatever the source, so authenticating the +server would buy nothing the verification does not already give. + +Each way of not producing a record MUST be a distinct named `reason` — `invalid_epoch`, +`not_recorded`, `record_unreadable`, `unimplemented_ruleset` — never a zero, a default, or a +neighbouring epoch's record. The caller is deciding whether to re-census a week of chain history, +and the three refusals mean "ask someone else", "this node is broken", and "your ruleset is newer +than mine". + +### 24.10. Adopting an epoch from peers + +Every dialled peer is untrusted (NC-12), and the requirement a record names is the amount this +operator posts as collateral. **A record from a peer is adopted because it is recomputable, never +because the peer is trusted.** + +A receiving node MUST re-derive a candidate from a predecessor it already holds, through +`EpochRecord::advance`, and MUST require every field of the result to match — not only the +requirement. It MUST NOT restate the model's arithmetic to perform that check. Consequently a peer +cannot lie about any derived quantity while keeping its census inputs, cannot lie about the ruleset, +and cannot skip an epoch. + +A receiving node CANNOT check the census inputs against the chain. A record whose arithmetic is +impeccable and whose inputs are fiction is indistinguishable from an honest one by re-derivation +alone. The sample is the only defence against that residue, and it is bounded: + +* The sample MUST be sized by `dig_mirror_collateral::sync_sample_plan` against a chain-derived + owner population. A node that does not know the population MUST NOT adopt; a sample drawn from an + unknown population supports no confidence claim. +* Below `SYNC_MIN_POPULATION` the plan is advisory and the node MUST derive from chain instead. +* Adoption requires the plan's strict two-thirds agreement threshold, tallied over the FULL record. + A plurality MUST NOT be adopted: the plurality is what an attacker holding a minority of + identities is trying to produce. +* The threshold is a supermajority **of the planned sample**, and the plan caps `sample_size`, so + the threshold does not grow with the population. A node MUST therefore refuse a sample with more + distinct responders than `sample_size`, whole and untrimmed. Counting a fixed threshold against a + larger responder set would adopt a plurality, which the clause above forbids. +* More distinct responders than the chain-derived population is a detectable identity lie and MUST + refuse the whole sample rather than trim it. +* A responder that answers twice, differently, MUST have both answers discarded. +* Every epoch after the first is produced by a census, so a candidate for an epoch greater than 1 + that carries no census height MUST be refused. A node MUST NOT treat an absent height as a check + that does not apply, which would let a responder opt out of the advancing requirement by omitting + a field. +* The census height is this node's bookkeeping and is excluded from the tally key, so agreeing + responders may still differ on it. A node MUST carry the LOWEST height offered by the agreeing + cohort. Taking any responder-chosen height would let one member of an honest cohort name a height + no later census can advance past, denying every subsequent epoch. **This clause and the + missing-height refusal above are only correct together**: an absent height orders below every + present one, so taking the lowest without refusing absent heights would let one responder strip + the height from an honest cohort and disable the next epoch's advancing check instead. + +**A known limitation, stated rather than assumed away:** the plan counts distinct collateralised +owners, while a node samples distinct peers, and a peer's owner attribution is not proven on this +path. One adversary holding many peer identities therefore looks like many owners to the sampler. +This is why adoption is never load-bearing — the sample buys the ability to skip an expensive +historical re-derivation, never the right to be wrong — and why a node that can census an epoch +itself MUST prefer its own computation. + +Preferring its own computation is a requirement on the STORE, and it binds precisely where the two +disagree. §24.9 makes a held record immutable so that no peer can walk a node off the network's +history; that immutability MUST NOT also prevent a node correcting itself. A record held with +`AdoptedFromPeers` provenance MUST be superseded by one this node censused for the same epoch, even +when the two records differ. No other pair may supersede, and any other differing record remains a +conflict. + +**What keeps a peer off the superseding side is a discipline, not a type.** `StoredRecord` carries +its provenance as an ordinary deserialisable field, so a wire record naming `censused` decodes as +`Censused` and would supersede if it were handed to the store unchanged. It is not: the adoption +path DISCARDS whatever provenance a responder sent and stamps `AdoptedFromPeers` from its own tally. +Any future path that admits a record from the network MUST do the same. A node MUST NOT treat a +record's own claim about its provenance as evidence of that provenance. + +**A second stated limitation:** re-derivation is only as sound as the oracle it runs through. This +node trusts `dig_mirror_collateral::EpochRecord::advance` to be the network's arithmetic, and checks +a candidate by comparing against what that function produces. A defect in `advance` is therefore not +detectable on this path — it would be reproduced identically by every node that verified through it. +The mitigation is that `advance` is the single published implementation the whole network derives +from, so a divergence is a release-level event rather than a per-peer one. + +### 24.11. Retention + +Retention is **off by default**: a node keeps every epoch forever unless the operator sets +`retention_epochs` in `collateral.json`. The model's premise is that any node can recompute any past +epoch and reach the same answer, so a default that discarded history would erode that on every node +at once. + +`retention_epochs` counts back from the current epoch inclusive. A value of `0` MUST read as the +default rather than as "keep nothing", which would discard the epoch currently in force. **Epoch 1 +MUST NOT be pruned under any policy**: it is the base case every verification walk terminates at, so +a node that discarded it could no longer check anything a peer offered it. diff --git a/crates/dig-node-service/src/collateral.rs b/crates/dig-node-service/src/collateral.rs index 42df655..78d7c9f 100644 --- a/crates/dig-node-service/src/collateral.rs +++ b/crates/dig-node-service/src/collateral.rs @@ -38,6 +38,15 @@ const COLLATERAL_CONFIG_FILE: &str = "collateral.json"; /// The file holding the per-epoch records this node has censused, one JSON record per line. const EPOCH_RECORD_FILE: &str = "collateral-epochs.jsonl"; +/// The one-based first epoch, which no retention policy may discard. +/// +/// Named rather than written as `1` at the one place it is used, because what makes it exempt is +/// not that it is small — it is that it is the base case the recurrence unrolls from. +/// +/// It is also the one epoch with no census height, which is why [`crate::collateral_sync::verify`] +/// needs the same name to refuse a missing height on any LATER epoch. +pub(crate) const GENESIS_EPOCH: u64 = 1; + /// This node's local collateral preferences. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] pub struct CollateralConfig { @@ -50,6 +59,21 @@ pub struct CollateralConfig { /// cushion they were never offered. #[serde(default = "default_margin_bp")] pub margin_bp: u64, + + /// How many epochs of history to keep, or `None` to keep everything. + /// + /// **`None` is the default, and the default is the point.** The model's premise is that any + /// node can recompute any past epoch and reach the same answer, so a node that discarded + /// history by default would quietly erode the network's ability to audit itself — and it would + /// do so on every node at once, since a default is what almost every operator runs. Truncation + /// is a deliberate choice about one's own disk, made once, and never made on an operator's + /// behalf. + /// + /// `default` rather than required for the same reason `margin_bp` is: a config written before + /// this field existed expressed no retention preference, and reading one into it would be + /// inventing a decision the operator never made. + #[serde(default)] + pub retention_epochs: Option, } fn default_margin_bp() -> u64 { @@ -60,11 +84,30 @@ impl Default for CollateralConfig { fn default() -> Self { CollateralConfig { margin_bp: SAFETY_MARGIN_BP_DEFAULT, + // Keep everything. See the field's own documentation for why this is not a tuning + // choice. + retention_epochs: None, } } } impl CollateralConfig { + /// The retention policy this configuration expresses. + /// + /// A named method rather than a field read at each call site, so that "no preference means + /// keep everything" is stated once instead of being re-decided by whoever is holding the + /// `Option` — which is how a default comes to differ between two surfaces. + #[must_use] + pub fn retention(&self) -> RetentionPolicy { + match self.retention_epochs { + // Zero is not "keep nothing"; it is a value no operator can act on, and honouring it + // would delete the whole history including the epoch currently in force. It reads as + // the default, which is the direction that cannot lose data. + None | Some(0) => RetentionPolicy::KeepEverything, + Some(epochs) => RetentionPolicy::KeepEpochs(epochs), + } + } + /// Load from the node's own machine-wide state directory. /// /// The production entry point. It resolves the directory ITSELF via [`crate::state::state_dir`] @@ -138,6 +181,199 @@ impl CollateralConfig { } } +/// How this node came to hold a record. +/// +/// Carried, and displayed, because the three ways differ in **what was verified**, and an operator +/// reading a requirement is entitled to know which one produced it. A bootstrap record depends on +/// nothing and cannot be wrong. A censused record rests on this node's own chain reads. An adopted +/// one rests on a sample of untrusted peers whose *arithmetic* this node re-derived but whose +/// *census inputs* it did not check against chain — the weakest of the three, and the one that +/// would be most damaging to present with the authority of the strongest. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum RecordProvenance { + /// Epoch 1, derived from nothing: the anchor the recurrence unrolls from. + Bootstrap, + /// Censused by this node from its own chain reads. + Censused, + /// Adopted from a sample of peers, every counted response re-derived first. + AdoptedFromPeers { + /// How many distinct owners returned the record that was adopted. + agreed: u64, + /// How many distinct owners answered at all. + sampled: u64, + }, +} + +impl RecordProvenance { + /// How strong the evidence behind a record is, for comparison only. + /// + /// Used in exactly one place — deciding whether a re-`put` of an IDENTICAL record may record + /// stronger evidence for it (see [`EpochRecordStore::put`]). It never orders two different + /// records, because a record is not made true by the confidence of whoever offered it. + fn strength(self) -> u8 { + match self { + RecordProvenance::AdoptedFromPeers { .. } => 0, + RecordProvenance::Censused => 1, + RecordProvenance::Bootstrap => 2, + } + } +} + +/// Whether a record `held` may be replaced by a DIFFERING record offered with provenance +/// `incoming`. +/// +/// True in exactly one direction: this node's own census superseding what it had adopted from a +/// sample of peers. That is the case the prefer-own-computation rule exists for, and — because +/// `held.record != incoming.record` is what brings a caller here — it is reachable only when the +/// two disagree, which is the only case where superseding changes an answer. +/// +/// Deliberately not expressed as a `strength()` comparison. Strength orders EVIDENCE for one +/// record; this orders two different records, which strength explicitly refuses to do. +/// +/// # What keeps a peer off the `incoming` side, stated exactly +/// +/// Not the type. [`RecordProvenance`] is an ordinary deserialisable field, so a wire record naming +/// `censused` decodes as [`RecordProvenance::Censused`] and WOULD supersede here if it reached this +/// function unchanged. What prevents that is [`crate::collateral_sync::adopt`], which discards the +/// provenance a responder sent and stamps [`RecordProvenance::AdoptedFromPeers`] from its own +/// tally. +/// +/// So this is a discipline held in one function, and any future path that admits a record from the +/// network must hold it too. A caller that passes a deserialised peer record straight to +/// [`EpochRecordStore::put`] reopens the hole this predicate exists to close. +fn own_census_supersedes(held: RecordProvenance, incoming: RecordProvenance) -> bool { + matches!(held, RecordProvenance::AdoptedFromPeers { .. }) + && matches!(incoming, RecordProvenance::Censused) +} + +/// One epoch as this node stores it: the consensus record, plus how this node came by it. +/// +/// # Why the census height is here and not in `EpochRecord` +/// +/// `EpochRecord` is the consensus object — the thing every node must derive identically, and the +/// thing `EpochRecord::advance` reproduces from its predecessor and a census. The height at which +/// the census was TAKEN is not one of its inputs: two nodes reading the same chain at the same +/// height derive the same record whether or not either records the height. It is carried here +/// because it is what makes a record auditable — it names the block a disputed census can be +/// re-run against — and because it must never be able to change the arithmetic. +/// +/// It is `Option` rather than required for one honest reason: epoch 1 is derived from nothing and +/// was taken at no height. `None` therefore means "no census was taken", not "the height was lost", +/// and [`RecordProvenance`] beside it says which case a reader is in. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub struct StoredRecord { + /// The consensus record, flattened onto the line. + /// + /// Flattened rather than nested so that a line written before this envelope existed — a bare + /// `EpochRecord` — still parses, with `census_height` absent and the provenance defaulted to + /// the weakest reading. The store had no production writer before dig-node#387, so no such + /// line exists in the wild; the compatibility costs one attribute and removes the class of + /// migration bug entirely. + #[serde(flatten)] + pub record: EpochRecord, + + /// The block height the census behind this record was taken at, absent at epoch 1. + #[serde(default)] + pub census_height: Option, + + /// How this node came by the record. + #[serde(default = "weakest_provenance")] + pub provenance: RecordProvenance, +} + +/// The provenance a line that does not state one is read as. +/// +/// The WEAKEST, deliberately. A line with no provenance is one this node cannot account for, and +/// reading it as `Censused` would upgrade an unexplained record to "I verified this myself". +fn weakest_provenance() -> RecordProvenance { + RecordProvenance::AdoptedFromPeers { + agreed: 0, + sampled: 0, + } +} + +impl StoredRecord { + /// Epoch 1: the record that depends on nothing. + #[must_use] + pub fn bootstrap() -> Self { + StoredRecord { + record: EpochRecord::bootstrap(), + census_height: None, + provenance: RecordProvenance::Bootstrap, + } + } + + /// A record this node censused itself, at `census_height`. + #[must_use] + pub fn censused(record: EpochRecord, census_height: u32) -> Self { + StoredRecord { + record, + census_height: Some(census_height), + provenance: RecordProvenance::Censused, + } + } + + /// Is this record governed by a ruleset this build implements? + /// + /// The **protocol-version ceiling check** (dig-node#387, D1's remedy). A record naming a + /// version above what this build knows is one this binary cannot interpret even though every + /// field of it parses: the arithmetic that produced it is not the arithmetic this build has. + /// Serving it as authoritative is how a forged record reached an 18,482,313.402 DIG + /// recommendation in a probe — the figure parsed, so nothing downstream questioned it. + /// + /// Delegated to `ProtocolVersion::implemented`, never to a comparison against a local + /// constant: the set of implemented versions is `dig-mirror-collateral`'s to state, and a + /// second copy of it here would be a rival implementation of the one fact that decides whether + /// this node speaks the network's rules. + #[must_use] + pub fn is_interpretable(&self) -> bool { + self.record.protocol_version.implemented().is_ok() + } +} + +/// What a [`put`](EpochRecordStore::put) did. +/// +/// A refusal is a RESULT here, not an error. History is immutable: a record that contradicts one +/// already held is refused, and the caller — a sampled sync adopting from peers — needs to tell +/// "already had it" from "a peer told me something different" in order to report the second. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PutOutcome { + /// The record was appended. + Written, + /// This exact record was already held, with evidence at least as strong. + AlreadyPresent, + /// A DIFFERENT record is already held for this epoch, and was kept. + /// + /// The stored one wins, always. A node that let the newest writer win could be walked off the + /// network's history one epoch at a time by whoever spoke last. + Conflict { + /// What this node holds, and continues to hold. + held: Box, + }, +} + +/// How much history to keep. +/// +/// **The default is [`RetentionPolicy::KeepEverything`]**, and the default is the point: the model's +/// whole premise is that any node can recompute any past epoch and reach the same answer, and a +/// node that discarded history by default would quietly erode the network's ability to audit +/// itself. Truncation is an operator's deliberate choice about their own disk. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RetentionPolicy { + /// Keep every epoch, forever. The default. + KeepEverything, + /// Keep the newest `epochs` epochs, counting back from the current one. + KeepEpochs(u64), +} + +/// A year of epochs, derived from the epoch length rather than stated as a number. +/// +/// `dig_constants::MIRROR_EPOCH_LENGTH_MS` is seven days, so this is 52. Writing `52` here would be +/// a second statement of the schedule that a retune of the epoch length would silently contradict. +pub const RETENTION_ONE_YEAR_EPOCHS: u64 = + (365 * 24 * 60 * 60 * 1_000) / (dig_constants::MIRROR_EPOCH_LENGTH_MS as u64); + /// The per-epoch collateral records this node has censused. /// /// Append-only JSONL, highest revision of an epoch winning, mirroring the spend-audit record's @@ -151,7 +387,7 @@ pub struct EpochRecordStore { #[derive(Debug, Clone, PartialEq, Eq)] pub enum StoredEpoch { /// A record was found and parsed. - Found(Box), + Found(Box), /// No record for this epoch. Absent, /// A record for this epoch exists and could not be read. @@ -181,8 +417,82 @@ impl EpochRecordStore { &self.path } - /// Append one censused record. - pub fn put(&self, record: &EpochRecord) -> std::io::Result<()> { + /// Record one epoch, refusing to contradict an epoch already held. + /// + /// **This is the store's production writer** (dig-node#387). Before it existed the store was + /// written only by its own tests, so `control.collateral.requirement` answered + /// `unknown / not_censused` on every node in the network, forever. + /// + /// # Historical records are permanent and immutable + /// + /// The model's premise is that any node can recompute any past epoch and get the same answer. + /// A store that let a later write replace an earlier one would break that at the only point + /// where it matters: an attacker who can get a record in front of this node could walk it off + /// the network's history one epoch at a time, and every downstream figure — including the + /// amount of $DIG this operator posts — would follow. So a record that DIFFERS from one + /// already held is refused as [`PutOutcome::Conflict`] and the held one is kept. + /// + /// An identical record offered with STRONGER evidence is appended, because the consensus + /// figures do not change — only this node's account of how it came by them improves, and an + /// operator who later censuses an epoch they had adopted from peers should see that. The + /// reverse is not appended: evidence does not weaken on re-offer. + /// + /// # This node's own census supersedes one adopted from peers, precisely when they disagree + /// + /// Immutability above is a defence against a PEER walking this node off the network's history, + /// and it must not also prevent this node correcting itself. A sampled adoption is explicitly + /// not load-bearing (see [`crate::collateral_sync`]): it buys the right to skip an expensive + /// re-derivation, never the right to be wrong. So when this node later censuses an epoch it had + /// adopted, and its own arithmetic DISAGREES with what the peers said, the census wins and the + /// record is superseded. + /// + /// Restricting that to `AdoptedFromPeers` → `Censused` is what keeps it from reopening the + /// hole. Every record reachable from the network carries `AdoptedFromPeers` provenance + /// ([`crate::collateral_sync::adopt`] stamps it, and it is the only provenance a peer answer + /// can acquire), so no peer — however many identities it holds — can satisfy this arm. A + /// disagreeing record from any other source is still a [`PutOutcome::Conflict`]. + /// + /// # Errors + /// + /// The underlying write, and an [`std::io::ErrorKind::InvalidData`] when the file holds a line + /// for this epoch that cannot be read. Refusing there is deliberate: appending beside a record + /// that cannot be compared would silently create two answers for one epoch. + pub fn put(&self, record: &StoredRecord) -> std::io::Result { + match self.get(record.record.epoch) { + StoredEpoch::Found(held) + if held.record != record.record + && !own_census_supersedes(held.provenance, record.provenance) => + { + return Ok(PutOutcome::Conflict { held }) + } + StoredEpoch::Found(held) + if held.provenance.strength() >= record.provenance.strength() => + { + return Ok(PutOutcome::AlreadyPresent) + } + StoredEpoch::Found(_) => {} + StoredEpoch::Unreadable => { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + format!( + "epoch {} is already recorded in a line that cannot be read; \ + refusing to write a second answer beside it", + record.record.epoch + ), + )) + } + StoredEpoch::Absent => {} + } + self.append(record)?; + Ok(PutOutcome::Written) + } + + /// Append one line, with no regard for what is already held. + /// + /// Private, and the only writer. [`Self::put`] owns the immutability decision; keeping the + /// bytes-to-disk step separate is what stops a second caller from acquiring the ability to + /// overwrite history by reaching past that decision. + fn append(&self, record: &StoredRecord) -> std::io::Result<()> { if let Some(dir) = self.path.parent() { crate::state::ensure_dir_restricted(dir)?; } @@ -217,11 +527,11 @@ impl EpochRecordStore { Err(e) if e.kind() == std::io::ErrorKind::NotFound => return StoredEpoch::Absent, Err(_) => return StoredEpoch::Unreadable, }; - let mut best: Option = None; + let mut best: Option = None; let mut saw_unreadable = false; for line in text.lines().filter(|l| !l.trim().is_empty()) { - match serde_json::from_str::(line) { - Ok(rec) if rec.epoch == epoch => { + match serde_json::from_str::(line) { + Ok(rec) if rec.record.epoch == epoch => { best = Some(rec); } Ok(_) => {} @@ -238,6 +548,96 @@ impl EpochRecordStore { None => StoredEpoch::Absent, } } + + /// Every epoch held, newest revision of each, in ascending epoch order. + /// + /// Unreadable lines are SKIPPED rather than reported here, and that is the one place in this + /// module where collapsing them is right: this is a listing, its caller is showing an operator + /// what the node holds, and [`Self::get`] remains the authority on any single epoch. A listing + /// that refused wholesale because one line of a thousand had rotted would hide the 999 that + /// are fine. + /// + /// # Errors + /// + /// The underlying read. A MISSING file is an empty list, not an error: a node that has never + /// recorded an epoch holds no epochs, which is a fact rather than a fault. + pub fn records(&self) -> std::io::Result> { + let text = match std::fs::read_to_string(&self.path) { + Ok(text) => text, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(e) => return Err(e), + }; + let mut by_epoch: std::collections::BTreeMap = + std::collections::BTreeMap::new(); + for line in text.lines().filter(|l| !l.trim().is_empty()) { + if let Ok(rec) = serde_json::from_str::(line) { + by_epoch.insert(rec.record.epoch, rec); + } + } + Ok(by_epoch.into_values().collect()) + } + + /// Apply a retention policy, as of `current_epoch`. Returns how many epochs were dropped. + /// + /// [`RetentionPolicy::KeepEverything`] — the default — reads nothing and writes nothing, so a + /// node that never opts in never rewrites this file at all. + /// + /// Truncation rewrites the file from the surviving records rather than editing it in place, so + /// the result is exactly one line per surviving epoch and a partially-written rewrite cannot + /// leave a half-line behind: the new content is written beside the old and renamed over it. + /// + /// # Errors + /// + /// The underlying read, write, or rename. + pub fn prune(&self, policy: RetentionPolicy, current_epoch: u64) -> std::io::Result { + let RetentionPolicy::KeepEpochs(keep) = policy else { + return Ok(0); + }; + // `keep` epochs INCLUDING the current one, so a policy of 1 keeps exactly the current + // epoch. Saturating, so a `keep` larger than the epoch number keeps everything rather than + // wrapping to a cutoff near `u64::MAX` that would discard the entire history. + let oldest_kept = current_epoch.saturating_sub(keep.saturating_sub(1)); + let held = self.records()?; + // Epoch 1 is NEVER pruned, whatever the policy says. It is the base case the whole + // recurrence unrolls from: every peer record this node verifies is checked against a chain + // of predecessors that terminates there, so a node that discarded it to save one line + // could no longer verify anything it was offered. Found by running a two-epoch retention on + // a real node, which dropped it and left the store empty. + let (kept, dropped): (Vec<_>, Vec<_>) = held + .into_iter() + .partition(|rec| rec.record.epoch >= oldest_kept || rec.record.epoch == GENESIS_EPOCH); + if dropped.is_empty() { + return Ok(0); + } + let mut body = Vec::new(); + for rec in &kept { + body.extend_from_slice(&serde_json::to_vec(rec).map_err(std::io::Error::other)?); + body.push(b'\n'); + } + let temp = self.path.with_extension("jsonl.rewrite"); + std::fs::write(&temp, &body)?; + crate::control::restrict_permissions(&temp); + std::fs::rename(&temp, &self.path)?; + crate::control::restrict_permissions(&self.path); + Ok(dropped.len() as u64) + } +} + +/// Write epoch 1's record if this node holds none, and report whether it did. +/// +/// The genesis half of the production writer, and the only record any node can produce with no +/// chain access and no peers: [`EpochRecord::bootstrap`] depends on nothing. It is what makes the +/// recurrence well-founded, and it is what a fresh node's sampled sync verifies its first peer +/// answer against. +/// +/// Idempotent, and safe against a store that already holds a DIFFERENT epoch-1 record: [`put`] +/// refuses the contradiction and this reports it rather than overwriting. +/// +/// # Errors +/// +/// The underlying write. +pub fn ensure_bootstrap(store: &EpochRecordStore) -> std::io::Result { + store.put(&StoredRecord::bootstrap()) } /// Does an unparseable line claim to be about `epoch`? @@ -321,11 +721,10 @@ pub fn current_epoch_now() -> CurrentEpoch { /// plausibility bound derived here would be a rival implementation of the controller /// `dig-mirror-collateral` owns, which is how two surfaces come to disagree about one price. /// -/// The honest remedy is a check the record can make against its OWN published invariants — chiefly -/// that `protocol_version` does not exceed what this build implements, since a record from a newer -/// model is one this build cannot interpret even when every field parses. That belongs with the -/// census writer, where those invariants live, and is tracked on dig-node#387 rather than bolted -/// on at the read side. +/// The one check the record CAN make against its own published invariants is made, below: a +/// record whose `protocol_version` exceeds what this build implements is refused rather than +/// served, because a record from a newer model is one this build cannot interpret even when every +/// field of it parses. /// /// The margin is deliberately not consulted. `required_per_store_dig_base_units` is the PRE-margin, /// consensus-derived figure every node derives identically; folding a local preference into it here @@ -341,17 +740,32 @@ pub fn requirement(store: &EpochRecordStore, current: CurrentEpoch) -> Collatera CurrentEpoch::NoChainSource => return unknown(CollateralUnknownReason::NoChainSource), }; match store.get(epoch) { + // THE PROTOCOL-VERSION CEILING (dig-node#387, D1's remedy). A record naming a ruleset this + // build does not implement is refused BEFORE its figures are read, not after: every field + // of such a record parses, so nothing downstream would question the number, and a forged + // one drove an 18,482,313.402 DIG recommendation in a probe. `RecordUnreadable` is the + // honest reason — the node holds a record for the epoch and cannot READ it, in the exact + // sense that matters, which is that it does not have the arithmetic that produced it. + StoredEpoch::Found(rec) if !rec.is_interpretable() => { + tracing::warn!( + epoch = rec.record.epoch, + protocol_version = rec.record.protocol_version.0, + "the stored collateral record names a ruleset this build does not implement; \ + refusing to serve its figures" + ); + unknown(CollateralUnknownReason::RecordUnreadable) + } StoredEpoch::Found(rec) => CollateralRequirementResult::Known { - epoch: rec.epoch, + epoch: rec.record.epoch, // The version that COMPUTED the epoch, read off the record, never the newest version // this build implements. The two differ exactly when a node has upgraded mid-schedule, // which is the one case where the client needs to know the difference. - protocol_version: rec.protocol_version.0, - required_per_store_dig_base_units: rec.required_per_store_dig_base_units, - stores: rec.census.stores, - owners: rec.census.owners, - multiplier_micros: rec.multiplier_micros, - handicap_dig_base_units: rec.handicap_dig_base_units, + protocol_version: rec.record.protocol_version.0, + required_per_store_dig_base_units: rec.record.required_per_store_dig_base_units, + stores: rec.record.census.stores, + owners: rec.record.census.owners, + multiplier_micros: rec.record.multiplier_micros, + handicap_dig_base_units: rec.record.handicap_dig_base_units, }, StoredEpoch::Absent => unknown(CollateralUnknownReason::NotCensused), StoredEpoch::Unreadable => unknown(CollateralUnknownReason::RecordUnreadable), @@ -567,6 +981,275 @@ mod tests { EpochRecordStore::at(dir.join(EPOCH_RECORD_FILE)) } + #[test] + fn a_record_that_contradicts_a_held_epoch_is_refused_and_the_held_one_kept() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + let held = StoredRecord::censused(record(9, 1_000_000, 1_000, 40), 7_000); + assert_eq!(store.put(&held).expect("put"), PutOutcome::Written); + + // A DIFFERENT record for the same epoch: the shape of an attacker walking a node off the + // network's history one epoch at a time. It differs in the requirement, which is the field + // that decides how much $DIG this operator posts. + let contradiction = StoredRecord::censused(record(9, 500_000, 600, 25), 7_001); + assert_ne!(contradiction.record, held.record, "the fixture must differ"); + match store.put(&contradiction).expect("put") { + PutOutcome::Conflict { held: kept } => assert_eq!(kept.record, held.record), + other => panic!("a contradiction must be refused, got {other:?}"), + } + // And the refusal is not merely reported — the store still answers with the ORIGINAL. + match store.get(9) { + StoredEpoch::Found(rec) => assert_eq!(rec.record, held.record), + other => panic!("the held record must survive, got {other:?}"), + } + } + + /// GATING finding 2 (dig-node#398 round 2), written as the exploit. + /// + /// The prefer-own-census rule was guarded behind `held.record != record.record`, so it was + /// reachable only when the peers and this node already AGREED — it held in every case except + /// the one it exists for. Combined with finding 1 that made an adopted lie permanent: nothing + /// this node could later compute would ever displace it. + /// + /// Two hops are exercised on purpose. Asserting only that the second `put` returns `Written` + /// would pass for an implementation that appended without superseding, so the stored answer is + /// read back and compared to the census. + #[test] + fn this_nodes_own_census_supersedes_a_peer_adopted_record_they_disagree_with() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + let adopted = StoredRecord { + record: record(9, 500_000, 600, 25), + census_height: Some(7_000), + provenance: RecordProvenance::AdoptedFromPeers { + agreed: 7, + sampled: 9, + }, + }; + assert_eq!(store.put(&adopted).expect("put"), PutOutcome::Written); + + // What this node computes for itself, DISAGREEING with what it adopted. The requirement is + // higher, so believing the peers would have left this operator under-collateralised. + let censused = StoredRecord::censused(record(9, 1_000_000, 1_000, 40), 7_001); + assert_ne!( + censused.record, adopted.record, + "the fixture must DISAGREE, or it exercises the identical-record path instead" + ); + assert!( + censused.record.required_per_store_dig_base_units + > adopted.record.required_per_store_dig_base_units + ); + assert_eq!(store.put(&censused).expect("put"), PutOutcome::Written); + match store.get(9) { + StoredEpoch::Found(rec) => { + assert_eq!(rec.record, censused.record, "the census must win"); + assert_eq!(rec.provenance, RecordProvenance::Censused); + } + other => panic!("expected the censused record, got {other:?}"), + } + } + + /// The other half of the same rule, and the reason it is not a `strength()` comparison: a peer + /// still cannot overwrite anything. Every record reachable from the network carries + /// `AdoptedFromPeers`, which is never on the winning side of `own_census_supersedes`. + #[test] + fn a_peer_still_cannot_overwrite_a_record_this_node_censused() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + let censused = StoredRecord::censused(record(9, 1_000_000, 1_000, 40), 7_000); + assert_eq!(store.put(&censused).expect("put"), PutOutcome::Written); + + let from_peers = StoredRecord { + record: record(9, 500_000, 600, 25), + census_height: Some(7_001), + provenance: RecordProvenance::AdoptedFromPeers { + agreed: 9, + sampled: 9, + }, + }; + match store.put(&from_peers).expect("put") { + PutOutcome::Conflict { held } => assert_eq!(held.record, censused.record), + other => panic!("a peer must not displace this node's census, got {other:?}"), + } + // And an adopted record cannot displace another adopted record either: the permitted + // direction is named, not merely "stronger evidence wins". + let second_peer_answer = StoredRecord { + record: record(9, 400_000, 500, 20), + census_height: Some(7_002), + provenance: RecordProvenance::AdoptedFromPeers { + agreed: 8, + sampled: 9, + }, + }; + let fresh = tempfile::tempdir().expect("tempdir"); + let store = store_at(fresh.path()); + assert_eq!(store.put(&from_peers).expect("put"), PutOutcome::Written); + assert!(matches!( + store.put(&second_peer_answer).expect("put"), + PutOutcome::Conflict { .. } + )); + } + + #[test] + fn an_identical_record_records_stronger_evidence_but_never_weaker() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + let figures = record(5, 900_000, 700, 30); + let adopted = StoredRecord { + record: figures, + census_height: Some(6_000), + provenance: RecordProvenance::AdoptedFromPeers { + agreed: 7, + sampled: 9, + }, + }; + let censused = StoredRecord::censused(figures, 6_000); + + assert_eq!(store.put(&adopted).expect("put"), PutOutcome::Written); + // Censusing an epoch previously adopted from peers is a genuine improvement in what this + // node can vouch for, and the consensus figures do not move. + assert_eq!(store.put(&censused).expect("put"), PutOutcome::Written); + match store.get(5) { + StoredEpoch::Found(rec) => assert_eq!(rec.provenance, RecordProvenance::Censused), + other => panic!("expected the censused record, got {other:?}"), + } + // The reverse is not recorded: evidence does not weaken on re-offer. Without this half the + // test would pass for a store that simply takes the newest line. + assert_eq!( + store.put(&adopted).expect("put"), + PutOutcome::AlreadyPresent + ); + match store.get(5) { + StoredEpoch::Found(rec) => assert_eq!(rec.provenance, RecordProvenance::Censused), + other => panic!("expected the censused record, got {other:?}"), + } + } + + #[test] + fn a_config_that_expresses_no_retention_keeps_everything() { + let dir = tempfile::tempdir().expect("tempdir"); + std::fs::write(dir.path().join(COLLATERAL_CONFIG_FILE), b"{}").expect("write"); + assert_eq!( + CollateralConfig::load_from(dir.path()).retention(), + RetentionPolicy::KeepEverything, + "retention is OFF by default; a node keeps everything unless told otherwise" + ); + // Zero is not a retention of nothing. Honouring it would delete the epoch in force. + assert_eq!( + CollateralConfig { + margin_bp: 0, + retention_epochs: Some(0) + } + .retention(), + RetentionPolicy::KeepEverything + ); + } + + #[test] + fn keep_everything_drops_nothing_and_a_retention_drops_only_what_falls_outside_it() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + for epoch in 1..=10u64 { + store + .put(&StoredRecord::censused( + record(epoch, 1_000_000, 100 + epoch, 10 + epoch), + 1_000 + epoch as u32, + )) + .expect("put"); + } + + // The default is the control: without it this test cannot tell a correct policy from one + // that truncates regardless. + assert_eq!( + store + .prune(RetentionPolicy::KeepEverything, 10) + .expect("prune"), + 0 + ); + assert_eq!(store.records().expect("records").len(), 10); + + // KeepEpochs(3) as of epoch 10 keeps 8, 9 and 10 — the bound is pinned from BOTH sides: + // epoch 8 must survive and epoch 7 must not, so an off-by-one in either direction fails. + // Epoch 1 survives regardless: it is the base case every verification walk starts from, + // and a node that discarded it could no longer check anything a peer offered it. So 6 of + // the 7 outside the window go, not all 7. + assert_eq!( + store.prune(RetentionPolicy::KeepEpochs(3), 10).expect("p"), + 6 + ); + let kept: Vec = store + .records() + .expect("records") + .iter() + .map(|rec| rec.record.epoch) + .collect(); + assert_eq!(kept, vec![1, 8, 9, 10]); + // The survivors are still whole records, not truncated lines. + match store.get(8) { + StoredEpoch::Found(rec) => assert_eq!(rec.census_height, Some(1_008)), + other => panic!("epoch 8 must survive intact, got {other:?}"), + } + assert_eq!(store.get(7), StoredEpoch::Absent); + } + + #[test] + fn a_record_from_an_unimplemented_ruleset_is_not_served_as_a_requirement() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + // The honest record for the same epoch is the control. It must answer `Known`, or this + // test would pass for a `requirement` that refuses everything. + let honest = StoredRecord::censused(record(6, 1_000_000, 1_000, 40), 5_000); + store.put(&honest).expect("put"); + assert!(matches!( + requirement(&store, CurrentEpoch::Final(6)), + CollateralRequirementResult::Known { .. } + )); + + // Same epoch, same figures, a ruleset this build does not implement. Every field parses, + // which is exactly why nothing downstream would question the number: a forged record of + // this shape drove an 18,482,313.402 DIG recommendation in a probe. + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + let mut forged = honest; + forged.record.protocol_version = dig_mirror_collateral::ProtocolVersion(u16::MAX); + store.put(&forged).expect("put"); + assert_eq!( + requirement(&store, CurrentEpoch::Final(6)), + CollateralRequirementResult::Unknown { + reason: CollateralUnknownReason::RecordUnreadable + }, + "a record this build cannot interpret is unknown, never a figure" + ); + } + + #[test] + fn the_genesis_record_is_written_once_and_is_idempotent() { + let dir = tempfile::tempdir().expect("tempdir"); + let store = store_at(dir.path()); + // Before: the node has censused nothing, and says so. + assert_eq!( + requirement(&store, CurrentEpoch::Final(1)), + CollateralRequirementResult::Unknown { + reason: CollateralUnknownReason::NotCensused + } + ); + assert_eq!( + ensure_bootstrap(&store).expect("bootstrap"), + PutOutcome::Written + ); + // After: a real answer, from a record derived from nothing. + assert!(matches!( + requirement(&store, CurrentEpoch::Final(1)), + CollateralRequirementResult::Known { epoch: 1, .. } + )); + // A node restarts more than once, and a second line for epoch 1 would be a second answer. + assert_eq!( + ensure_bootstrap(&store).expect("bootstrap"), + PutOutcome::AlreadyPresent + ); + assert_eq!(store.records().expect("records").len(), 1); + } + #[test] fn a_config_predating_the_margin_field_loads_as_the_default_not_zero() { let dir = tempfile::tempdir().expect("tempdir"); @@ -585,9 +1268,12 @@ mod tests { let dir = tempfile::tempdir().expect("tempdir"); // Deliberately not the default: a save/load that silently discarded the value would still // pass if the fixture used the default. - CollateralConfig { margin_bp: 250 } - .save_to(dir.path()) - .expect("save"); + CollateralConfig { + margin_bp: 250, + retention_epochs: None, + } + .save_to(dir.path()) + .expect("save"); assert_eq!(CollateralConfig::load_from(dir.path()).margin_bp, 250); } @@ -597,8 +1283,15 @@ mod tests { let store = store_at(dir.path()); // Two epochs present, with DIFFERENT multipliers and owner counts, so answering with the // wrong one is observable. A single-record fixture could not see that. - store.put(&record(7, 1_000_000, 1_000, 40)).expect("put"); - store.put(&record(8, 500_000, 600, 25)).expect("put"); + store + .put(&StoredRecord::censused( + record(7, 1_000_000, 1_000, 40), + 5_100, + )) + .expect("put"); + store + .put(&StoredRecord::censused(record(8, 500_000, 600, 25), 5_200)) + .expect("put"); let answer = requirement(&store, CurrentEpoch::Final(8)); let CollateralRequirementResult::Known { @@ -629,7 +1322,12 @@ mod tests { fn each_missing_fact_gets_its_own_reason() { let dir = tempfile::tempdir().expect("tempdir"); let store = store_at(dir.path()); - store.put(&record(3, 1_000_000, 1_000, 10)).expect("put"); + store + .put(&StoredRecord::censused( + record(3, 1_000_000, 1_000, 10), + 4_000, + )) + .expect("put"); let reason = |c| match requirement(&store, c) { CollateralRequirementResult::Unknown { reason } => reason, @@ -700,7 +1398,12 @@ mod tests { let store = store_at(dir.path()); // A healthy neighbouring record, so the fixture keeps an honest control: a store that was // entirely corrupt could not show that the corruption was ATTRIBUTED to epoch 5. - store.put(&record(4, 1_000_000, 1_000, 10)).expect("put"); + store + .put(&StoredRecord::censused( + record(4, 1_000_000, 1_000, 10), + 4_100, + )) + .expect("put"); use std::io::Write as _; let mut f = std::fs::OpenOptions::new() .append(true) @@ -768,7 +1471,9 @@ mod tests { fn the_known_answer_carries_the_census_inputs_a_client_needs() { let dir = tempfile::tempdir().expect("tempdir"); let store = store_at(dir.path()); - store.put(&record(12, 800_000, 750, 31)).expect("put"); + store + .put(&StoredRecord::censused(record(12, 800_000, 750, 31), 6_000)) + .expect("put"); let wire = serde_json::to_value(requirement(&store, CurrentEpoch::Final(12))).expect("serialise"); diff --git a/crates/dig-node-service/src/collateral_sync.rs b/crates/dig-node-service/src/collateral_sync.rs new file mode 100644 index 0000000..85b3e00 --- /dev/null +++ b/crates/dig-node-service/src/collateral_sync.rs @@ -0,0 +1,928 @@ +//! Adopting an epoch history from peers, and serving one to them. +//! +//! # The one sentence this module exists to make true +//! +//! **A record from a peer is adopted because it is RECOMPUTABLE, never because the peer is +//! trusted.** NC-12 binds every dialled peer as untrusted, and the value being fetched here is +//! consensus-adjacent in the most direct way possible: the requirement a record names is the amount +//! of $DIG this operator posts as collateral. A peer that can move it down leaves this node's +//! stores uncollateralised and silently unpaid; a peer that can move it up locks the operator's +//! money for nothing. The user's stated threat is specifically the *down* direction — nodes +//! spamming invalid mirror coins to force the requirement down — and a sampled sync that took a +//! peer's word would be a second, cheaper path to the same end. +//! +//! # What a receiving node CAN verify, and does +//! +//! A record carries its own census inputs, so the whole derivation is reproducible. This node +//! **re-derives** the candidate from a predecessor it already holds, via +//! [`EpochRecord::advance`](dig_mirror_collateral::EpochRecord::advance), and demands that every +//! field of the result match — not just the headline requirement. Nothing here restates the +//! formula: `equilibrium x multiplier - handicap` omits the floor clamp, and a check written that +//! way would accept records the network rejects. +//! +//! That means a peer cannot lie about **any derived quantity** — the requirement, the multiplier, +//! the handicap, the base price, the band, the signals — while keeping its census inputs. It also +//! cannot lie about the **ruleset**: `advance` refuses a protocol version this build does not +//! implement, on the candidate and on the seed, so a forged record from a "newer model" is refused +//! rather than believed. And it cannot lie about the **epoch**: the recurrence is defined only for +//! consecutive epochs, so a record is only ever verified against its immediate predecessor, walking +//! forward from a bootstrap record that depends on nothing. +//! +//! # What it CANNOT verify, and what it does about that +//! +//! **It cannot check the census inputs against the chain.** A peer that reports a smaller +//! `stores` count than the chain holds has produced a record whose arithmetic is impeccable and +//! whose inputs are fiction. Re-derivation cannot see that; only a chain read can. +//! +//! So the sample is the defence against exactly that residue, and it is bounded honestly: +//! +//! * The sample is sized by [`dig_mirror_collateral::sync_sample_plan`] against a **chain-derived +//! population**. A node that does not know the population does not get a plan — it gets +//! [`AdoptOutcome::Advisory`], and derives from chain instead. That is not a degraded mode to be +//! worked around; a sample drawn from an unknown population supports no confidence claim at all. +//! * Agreement is a strict two-thirds supermajority of a bounded sample, and its confidence number +//! is conditional on at most a fifth of the population being dishonest. Below +//! `SYNC_MIN_POPULATION` the plan says `advisory_only` and this node does not adopt. +//! * Hearing from **more distinct owners than the chain says exist** is not noise, it is a +//! detectable lie, and it refuses the whole sample rather than the excess responses. +//! * The threshold is a supermajority **of the planned sample**, and the plan CAPS the sample at 9 +//! — so the threshold is a fixed 7 whether the population is 20 or 20,000. Counting that 7 +//! against a larger responder set would turn a supermajority into a plurality, so more +//! responders than the plan drew for refuses the whole sample too, for the same reason. +//! * **Disagreement never resolves to a majority of a tiny sample.** No supermajority means +//! [`AdoptOutcome::NoAgreement`], which is an `unknown` with a reason — never a best guess, never +//! the most popular answer, and never a neighbouring epoch's figure. +//! +//! # The honest gap, stated rather than papered over +//! +//! The plan counts distinct **collateralised owners**; this node samples distinct **peers**, and a +//! peer's claimed owner attribution is not proven on this path. One adversary holding many peer +//! identities therefore looks like many owners to the sampler, which is precisely the assumption +//! the confidence figure rests on. This is why adoption is never load-bearing: the sample buys the +//! ability to SKIP an expensive historical re-derivation, never the right to be wrong. A node that +//! can census an epoch itself must prefer its own computation. [`crate::collateral::put`] is where +//! that is enforced, and it binds precisely where the two DISAGREE: a record held as +//! `AdoptedFromPeers` is superseded by one this node censused for the same epoch even when the +//! figures differ, while every other differing pair remains a conflict. +//! +//! What keeps a responder off the superseding side is [`adopt`] below, which DISCARDS the +//! provenance a peer sent and stamps `AdoptedFromPeers` from its own tally — not the type, which +//! deserialises `censused` from the wire perfectly happily. It is a discipline in one function, and +//! any future path admitting a record from the network must hold it too. + +use std::collections::BTreeMap; + +use dig_mirror_collateral::{sync_sample_plan, SyncSamplePlan}; + +use crate::collateral::{RecordProvenance, StoredRecord, GENESIS_EPOCH}; + +/// One peer's answer for one epoch. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PeerRecord { + /// Who answered, as this node identifies them. Used only to count DISTINCT responders. + pub responder: String, + /// What they returned. + pub record: StoredRecord, +} + +/// Why a candidate record was not counted. +/// +/// Every rejection is named. A sample that discarded responses silently would report "no +/// agreement" for a network that agreed perfectly and one liar, which sends an operator to +/// diagnose the wrong thing. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Rejection { + /// The record does not describe the epoch after the one it was verified against. + NonSequential { + /// The epoch a record would have had to describe. + expected: u64, + /// The epoch it did describe. + found: u64, + }, + /// The record, or the predecessor it was checked against, names a ruleset this build does not + /// implement. + UnimplementedRuleset { + /// The version named. + protocol_version: u16, + }, + /// Re-deriving the record from its own census inputs did not reproduce it. + /// + /// The strongest signal available on this path: the peer's arithmetic disagrees with the + /// network's, which no honest node can do. + ArithmeticMismatch, + /// The census height does not advance past the predecessor's. + /// + /// A census for a later epoch is taken at a later block. A record claiming otherwise describes + /// a chain that ran backwards. + CensusHeightNotAdvancing { + /// The predecessor's height. + previous: u32, + /// The height claimed. + found: u32, + }, + /// The record is for an epoch a census produced, but carries no census height. + /// + /// Only genesis has no height. A later record without one cannot be checked for advancing, so + /// accepting it would let a peer opt out of that check by omitting a field. + CensusHeightMissing { + /// The epoch the record claimed. + epoch: u64, + }, + /// This responder had already answered, differently. Both answers are discarded. + /// + /// Discarding BOTH is deliberate. Keeping the first would let a peer that equivocates still + /// contribute a vote, and equivocation is the clearest evidence of dishonesty this sampler can + /// observe. + Equivocated, +} + +/// What a sampled sync concluded. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum AdoptOutcome { + /// The sample agreed; this record may be stored. + Adopted { + /// The record, with its adoption recorded in its provenance. + record: Box, + }, + /// The sample is advisory only: it informs, it does not decide. + /// + /// Either the population is unknown to this node, or it is below the threshold at which the + /// "at most a fifth dishonest" assumption every confidence figure rests on is meaningful. The + /// node derives the epoch from chain instead. + Advisory { + /// Why, in a sentence an operator can act on. + reason: &'static str, + }, + /// More distinct responders answered than the chain says owners exist. + /// + /// Not noise: a finite population is what makes the sample a sample, and exceeding it is + /// evidence of fabricated identities. The whole sample is refused, not trimmed — a prefix of an + /// attacker-writable set is a set the attacker chose. + PopulationExceeded { + /// The chain-derived population. + population: u64, + /// How many distinct responders answered. + responders: u64, + }, + /// More distinct responders answered than the plan drew for. + /// + /// The agreement threshold is a supermajority OF THE PLANNED SAMPLE — at any population of 20 + /// or more it is a fixed 7 of 9, because the sample size is capped. Counting that 7 against a + /// responder set larger than 9 turns a supermajority into a plurality, and 7 identities out of + /// a 1000-strong population would carry a record this node then treats as history. So a sample + /// larger than the one planned is refused whole, exactly as `PopulationExceeded` is: the + /// threshold is only meaningful against the set it was computed for. + SampleExceeded { + /// How many responders the plan drew for. + sample_size: u64, + /// How many distinct responders answered. + responders: u64, + }, + /// No record reached the agreement threshold. + /// + /// An `unknown` with a reason, never a best guess. The most popular answer in a sample that + /// failed to converge is exactly the answer an attacker with a minority of identities is trying + /// to produce. + NoAgreement { + /// How many responses survived verification. + verified: u64, + /// How many agreeing responses were needed. + needed: u64, + /// The largest number that agreed on any one record. + best: u64, + }, +} + +/// Verify a candidate against the predecessor this node already holds. +/// +/// The whole record is reproduced, not just its headline figure. Checking only +/// `required_per_store_dig_base_units` would accept a record whose multiplier and handicap are both +/// wrong in compensating directions — and the multiplier is what the buffer's escalation headroom +/// scales, so the compensation would not survive into the next epoch. +/// +/// # Errors +/// +/// A [`Rejection`] naming which check failed. +pub fn verify(prior: &StoredRecord, candidate: &StoredRecord) -> Result<(), Rejection> { + if !prior.is_interpretable() { + return Err(Rejection::UnimplementedRuleset { + protocol_version: prior.record.protocol_version.0, + }); + } + if !candidate.is_interpretable() { + return Err(Rejection::UnimplementedRuleset { + protocol_version: candidate.record.protocol_version.0, + }); + } + + // Re-derive. The candidate's OWN census inputs are the only thing taken from it; every derived + // field is this node's own arithmetic, through the crate that owns it. + let derived = prior + .record + .advance(candidate.record.census) + .map_err(|e| match e { + dig_mirror_collateral::CollateralError::NonSequentialEpoch { expected, found } => { + Rejection::NonSequential { expected, found } + } + _ => Rejection::UnimplementedRuleset { + protocol_version: candidate.record.protocol_version.0, + }, + })?; + if derived != candidate.record { + return Err(Rejection::ArithmeticMismatch); + } + + // A census for a later epoch is taken at a later block. + // + // Genesis is the one record with no height, because no census produced it. Every epoch from 2 + // on is the output of a census, so a record for one that carries no height is malformed — and + // it must be REFUSED rather than skipped. Matching on both heights being present made the + // guard opt-out: a peer omitting the field passed the check trivially, which is the weaker + // half of the same denial surface the tally's height choice closes. + match (prior.census_height, candidate.census_height) { + (Some(previous), Some(found)) if found <= previous => { + return Err(Rejection::CensusHeightNotAdvancing { previous, found }) + } + (_, None) if candidate.record.epoch > GENESIS_EPOCH => { + return Err(Rejection::CensusHeightMissing { + epoch: candidate.record.epoch, + }) + } + _ => {} + } + Ok(()) +} + +/// Decide whether a sample of peer answers may be adopted as the epoch after `prior`. +/// +/// `population` is the count of distinct collateralised owner hashes the chain reports at the +/// census height — `None` when this node cannot read it, which is not a detail to route around: +/// see [`AdoptOutcome::Advisory`]. +pub fn adopt( + prior: &StoredRecord, + population: Option, + responses: &[PeerRecord], +) -> AdoptOutcome { + let Some(population) = population else { + return AdoptOutcome::Advisory { + reason: "this node cannot read the chain-derived owner population, so a sample of \ + peers supports no confidence claim; derive the epoch from chain instead", + }; + }; + let plan: SyncSamplePlan = sync_sample_plan(population); + if plan.advisory_only { + return AdoptOutcome::Advisory { + reason: "the collateralised owner population is too small for a sample to mean \ + anything; derive the epoch from chain instead", + }; + } + + // One vote per responder, and an equivocating responder gets none. Recorded before any + // counting so that a peer cannot buy a second vote by answering twice. + let mut by_responder: BTreeMap<&str, Option<&StoredRecord>> = BTreeMap::new(); + for response in responses { + match by_responder.get(response.responder.as_str()) { + None => { + by_responder.insert(&response.responder, Some(&response.record)); + } + Some(Some(first)) if **first == response.record => {} + // Answered before, differently: both answers are discarded. + Some(_) => { + by_responder.insert(&response.responder, None); + } + } + } + + // A finite, chain-derived population is what makes this a sample. Hearing from more distinct + // responders than the chain says owners exist is a detectable lie about identity, and it + // refuses the sample outright rather than trimming it to a prefix the attacker chose. + let responders = by_responder.len() as u64; + if responders > plan.population { + return AdoptOutcome::PopulationExceeded { + population: plan.population, + responders, + }; + } + // ...and the same discipline against the SAMPLE, which is the set the threshold below was + // computed for. `plan.population` is the wrong bound to stop at: it can be in the thousands + // while `agreement_threshold` stays 7, so a check that only refuses on population lets 7 + // agreeing identities out of 1000 clear a bar that means "seven ninths". + if responders > plan.sample_size { + return AdoptOutcome::SampleExceeded { + sample_size: plan.sample_size, + responders, + }; + } + + // Verify, then tally by the FULL record. Two records that agree on the requirement but differ + // anywhere else are different answers, and counting them together would let a disagreement + // about the multiplier ride in on agreement about today's price. + let mut tally: BTreeMap = BTreeMap::new(); + let mut verified = 0u64; + for record in by_responder.values().flatten() { + if verify(prior, record).is_err() { + continue; + } + verified += 1; + // Keyed on the canonical JSON of the CONSENSUS record only. Provenance and census height + // are this node's bookkeeping about a peer, not part of what the network agrees on, so two + // peers that derived the same epoch must not be split into two camps by them. + let key = match serde_json::to_string(&record.record) { + Ok(key) => key, + Err(_) => continue, + }; + let entry = tally.entry(key).or_insert((0, **record)); + entry.0 += 1; + // The census height is not part of the tally key, so agreeing responders may still offer + // different heights — and one of them has to be carried forward. Take the LOWEST, never + // the first seen. + // + // "First seen" was first in `BTreeMap` order over a PEER-SUPPLIED responder id, so a + // single peer inside an otherwise honest cohort could name itself early, answer with the + // correct record and `census_height: u32::MAX`, and wedge every later epoch: `verify` + // requires the height to advance, and nothing advances past `u32::MAX`. That turns a guard + // into a denial primitive. + // + // The lowest is safe in the other direction because every tallied record passed `verify`, + // which already refuses any height that does not advance past the predecessor's. So the + // minimum is still strictly above the prior epoch's, and choosing it costs at most a + // slightly conservative floor for the next epoch's check. + // + // THIS AND THE `CensusHeightMissing` REFUSAL IN `verify` ARE ONLY CORRECT TOGETHER. Rust + // orders `None` BELOW every `Some`, so taking the minimum without that refusal would let + // one responder inside an honest cohort strip the height entirely — and an adopted record + // with no height silently disables the NEXT epoch's advancing check, which is the same + // denial surface reached from the other end. Do not remove either one believing they are + // independent. + if record.census_height < entry.1.census_height { + entry.1.census_height = record.census_height; + } + } + + let best = tally.values().map(|(count, _)| *count).max().unwrap_or(0); + if best < plan.agreement_threshold { + return AdoptOutcome::NoAgreement { + verified, + needed: plan.agreement_threshold, + best, + }; + } + let (agreed, winner) = tally + .into_values() + .max_by_key(|(count, _)| *count) + .expect("a non-zero best implies at least one tallied record"); + + AdoptOutcome::Adopted { + record: Box::new(StoredRecord { + record: winner.record, + census_height: winner.census_height, + provenance: RecordProvenance::AdoptedFromPeers { + agreed, + sampled: responders, + }, + }), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use dig_mirror_collateral::{EpochCensus, EpochRecord}; + + /// The bootstrap record, which every walk forward starts from. + fn genesis() -> StoredRecord { + StoredRecord::bootstrap() + } + + /// The HONEST successor of `prior` for a census of `stores` advertisements across `owners` + /// owners, taken at `height`. + /// + /// Derived through `advance` rather than hand-built, so a fixture cannot accidentally encode + /// arithmetic the crate does not actually produce — which would make every test below a test + /// of the fixture. + fn honest(prior: &StoredRecord, stores: u64, owners: u64, height: u32) -> StoredRecord { + let census = EpochCensus { + epoch: prior.record.epoch + 1, + stores, + owners, + locked: stores * 20_000, + }; + StoredRecord::censused( + prior.record.advance(census).expect("an honest successor"), + height, + ) + } + + fn sample(records: &[(&str, StoredRecord)]) -> Vec { + records + .iter() + .map(|(responder, record)| PeerRecord { + responder: (*responder).to_string(), + record: *record, + }) + .collect() + } + + /// A population comfortably at the sampling plateau, so the plan is a real plan (9 sampled, 7 + /// needed) rather than the degenerate advisory one. Chosen FROM `SYNC_MIN_POPULATION` rather + /// than picked because it looked big. + const PLATEAU_POPULATION: u64 = 40; + + #[test] + fn an_honest_successor_verifies() { + let prior = genesis(); + let candidate = honest(&prior, 120, 30, 5_000); + assert_eq!(verify(&prior, &candidate), Ok(())); + } + + #[test] + fn a_record_whose_requirement_was_edited_down_is_refused() { + let prior = genesis(); + let mut forged = honest(&prior, 120, 30, 5_000); + // The down direction is the user's stated threat: a requirement talked lower leaves this + // operator's stores uncollateralised while every surface reports success. + forged.record.required_per_store_dig_base_units -= 1; + assert_eq!( + verify(&prior, &forged), + Err(Rejection::ArithmeticMismatch), + "a peer's arithmetic must be re-derived, never believed" + ); + } + + #[test] + fn a_record_whose_multiplier_was_edited_is_refused_even_though_the_requirement_is_honest() { + let prior = genesis(); + let honest_next = honest(&prior, 120, 30, 5_000); + let mut forged = honest_next; + // The requirement is left EXACTLY as derived. Only the multiplier moves — the field the + // buffer's escalation headroom scales. A check that compared only the headline figure + // would pass this, and the lie would surface an epoch later as a wrong recommendation. + forged.record.multiplier_micros += 1; + assert_eq!( + forged.record.required_per_store_dig_base_units, + honest_next.record.required_per_store_dig_base_units, + "the fixture must differ ONLY in the multiplier, or it proves nothing about it" + ); + assert_eq!(verify(&prior, &forged), Err(Rejection::ArithmeticMismatch)); + } + + #[test] + fn a_record_from_an_unimplemented_ruleset_is_refused_rather_than_interpreted() { + let prior = genesis(); + let mut forged = honest(&prior, 120, 30, 5_000); + forged.record.protocol_version = dig_mirror_collateral::ProtocolVersion(u16::MAX); + assert_eq!( + verify(&prior, &forged), + Err(Rejection::UnimplementedRuleset { + protocol_version: u16::MAX + }) + ); + } + + #[test] + fn a_record_that_skips_an_epoch_is_refused() { + let prior = genesis(); + let next = honest(&prior, 120, 30, 5_000); + let two_ahead = honest(&next, 130, 31, 6_000); + assert_eq!( + verify(&prior, &two_ahead), + Err(Rejection::NonSequential { + expected: 2, + found: 3 + }), + "the recurrence is defined only for consecutive epochs" + ); + } + + #[test] + fn a_census_height_that_does_not_advance_is_refused() { + let prior = honest(&genesis(), 120, 30, 5_000); + let mut candidate = honest(&prior, 130, 31, 9_000); + candidate.census_height = Some(5_000); + assert_eq!( + verify(&prior, &candidate), + Err(Rejection::CensusHeightNotAdvancing { + previous: 5_000, + found: 5_000 + }), + "a later epoch is censused at a later block" + ); + } + + #[test] + fn an_unknown_population_is_advisory_and_never_adopts() { + let prior = genesis(); + let good = honest(&prior, 120, 30, 5_000); + // Nine honest, agreeing peers -- a sample that WOULD adopt if the population were known. + // The control matters: without it this test would pass for a function that never adopts. + let responses = sample(&[ + ("a", good), + ("b", good), + ("c", good), + ("d", good), + ("e", good), + ("f", good), + ("g", good), + ("h", good), + ("i", good), + ]); + assert!(matches!( + adopt(&prior, Some(PLATEAU_POPULATION), &responses), + AdoptOutcome::Adopted { .. } + )); + assert!(matches!( + adopt(&prior, None, &responses), + AdoptOutcome::Advisory { .. } + )); + } + + #[test] + fn one_liar_among_honest_peers_does_not_stop_adoption_and_does_not_get_counted() { + let prior = genesis(); + let good = honest(&prior, 120, 30, 5_000); + let mut liar = good; + liar.record.required_per_store_dig_base_units /= 2; + // Seven honest peers, one liar, one equivocator. Exactly ONE actor is dishonest per role, + // and the honest majority is a truthful control: an all-hostile fixture cannot see whether + // an honest answer would have been counted. + let mut responses = sample(&[ + ("a", good), + ("b", good), + ("c", good), + ("d", good), + ("e", good), + ("f", good), + ("g", good), + ("liar", liar), + ]); + responses.push(PeerRecord { + responder: "equivocator".to_string(), + record: good, + }); + responses.push(PeerRecord { + responder: "equivocator".to_string(), + record: liar, + }); + + match adopt(&prior, Some(PLATEAU_POPULATION), &responses) { + AdoptOutcome::Adopted { record } => { + assert_eq!(record.record, good.record, "the honest record was adopted"); + assert_eq!( + record.provenance, + RecordProvenance::AdoptedFromPeers { + agreed: 7, + sampled: 9 + }, + "the equivocator is sampled but never counted, and the liar never agrees" + ); + } + other => panic!("expected adoption, got {other:?}"), + } + } + + #[test] + fn a_sample_that_does_not_converge_is_unknown_rather_than_a_majority() { + let prior = genesis(); + let good = honest(&prior, 120, 30, 5_000); + // A DIFFERENT but internally honest record: a real disagreement about the census, which is + // the residue re-derivation cannot resolve. Both camps verify; neither reaches 7 of 9. + let other = honest(&prior, 121, 30, 5_000); + assert_ne!( + good.record, other.record, + "the two camps must really differ" + ); + let responses = sample(&[ + ("a", good), + ("b", good), + ("c", good), + ("d", good), + ("e", good), + ("f", other), + ("g", other), + ("h", other), + ("i", other), + ]); + assert_eq!( + adopt(&prior, Some(PLATEAU_POPULATION), &responses), + AdoptOutcome::NoAgreement { + verified: 9, + needed: 7, + best: 5 + }, + "a plurality is not a supermajority, and the plurality is what an attacker aims for" + ); + } + + #[test] + fn more_responders_than_the_chain_says_owners_exist_refuses_the_whole_sample() { + use dig_mirror_collateral::SYNC_MIN_POPULATION; + let prior = genesis(); + let good = honest(&prior, 120, 30, 5_000); + // Every response is HONEST and they all agree, so this refusal is about the identity claim + // alone; a fixture with a liar in it could not tell the two reasons apart. + // + // The population is taken FROM `SYNC_MIN_POPULATION` rather than picked, because below it + // the plan is advisory and `PopulationExceeded` is unreachable — a smaller fixture reports + // `Advisory` and would have pinned the wrong refusal. Both sides of the bound are checked: + // one responder over must be refused, and exactly at the population must not be. + let responders: Vec<(String, StoredRecord)> = (0..=SYNC_MIN_POPULATION) + .map(|i| (format!("peer-{i}"), good)) + .collect(); + let over: Vec = responders + .iter() + .map(|(responder, record)| PeerRecord { + responder: responder.clone(), + record: *record, + }) + .collect(); + assert_eq!( + over.len() as u64, + SYNC_MIN_POPULATION + 1, + "the over-the-bound sample must really exceed the population" + ); + assert_eq!( + adopt(&prior, Some(SYNC_MIN_POPULATION), &over), + AdoptOutcome::PopulationExceeded { + population: SYNC_MIN_POPULATION, + responders: SYNC_MIN_POPULATION + 1 + } + ); + + // Exactly AT the population is not a population excess — but it is still more responders + // than the plan drew for, and the round-2 gate on dig-node#398 showed why that must also + // refuse: the agreement threshold was computed for a sample of 9, so counting it against + // 20 responders would accept 7 of 20. This assertion previously expected `Adopted`, which + // encoded exactly that defect. + let at_population = &over[..SYNC_MIN_POPULATION as usize]; + assert_eq!( + adopt(&prior, Some(SYNC_MIN_POPULATION), at_population), + AdoptOutcome::SampleExceeded { + sample_size: 9, + responders: SYNC_MIN_POPULATION, + }, + "at the population but over the sample is a sample excess, not a population one" + ); + + // At the SAMPLE bound the same honest responses are adopted. Without this the test could + // not tell a correct guard from one that refuses every sample. + let at_sample = &over[..9]; + assert!( + matches!( + adopt(&prior, Some(SYNC_MIN_POPULATION), at_sample), + AdoptOutcome::Adopted { .. } + ), + "exactly at the planned sample is not an excess" + ); + } + + #[test] + fn the_epoch_one_record_needs_no_census_height_and_still_seeds_a_walk() { + let genesis = genesis(); + assert_eq!( + genesis.census_height, None, + "no census produced epoch 1, so there is no height to record" + ); + assert_eq!(genesis.provenance, RecordProvenance::Bootstrap); + assert_eq!(verify(&genesis, &honest(&genesis, 5, 2, 1_000)), Ok(())); + } + + #[test] + fn a_bare_epoch_record_line_reads_back_with_the_weakest_provenance() { + // A line written before the envelope existed: no provenance, no height. Reading it as + // `Censused` would upgrade an unaccounted-for record to "I verified this myself". + let bare = serde_json::to_string(&EpochRecord::bootstrap()).expect("serialize"); + let parsed: StoredRecord = serde_json::from_str(&bare).expect("a bare record still parses"); + assert_eq!(parsed.record, EpochRecord::bootstrap()); + assert_eq!(parsed.census_height, None); + assert_eq!( + parsed.provenance, + RecordProvenance::AdoptedFromPeers { + agreed: 0, + sampled: 0 + } + ); + } + + /// A population far above the plateau, where the capped sample makes the FIXED threshold of 7 + /// a plurality rather than a supermajority. Chosen from the plan's own arithmetic — anything + /// at or above `SYNC_MIN_POPULATION` caps `sample_size` at 9 — not because it looked large. + const POPULATION_ABOVE_THE_SAMPLE_CAP: u64 = 1_000; + + /// Two internally-consistent records that disagree about the money. + /// + /// Both pass `verify`: each is `advance`d from the same predecessor, so the arithmetic is + /// impeccable in both. They differ only in the CENSUS INPUT, which is the one thing a peer + /// supplies and this node cannot re-derive — the whole reason adoption is not load-bearing. + /// Only the OWNER count varies. `stores` and `locked` are held equal so the multiplier — and + /// with it the base price — is identical in both, leaving the handicap as the single moving + /// part. A fixture that varied several inputs at once could not say which one carried the lie. + fn truth_and_a_consistent_lie(prior: &StoredRecord) -> (StoredRecord, StoredRecord) { + let truth = honest(prior, 5_625, 40, 9_000); + let lie = honest(prior, 5_625, 2, 9_000); + assert_eq!(verify(prior, &truth), Ok(())); + assert_eq!( + verify(prior, &lie), + Ok(()), + "the lie must be VERIFIABLE, or the test proves only that verify works" + ); + assert!( + lie.record.required_per_store_dig_base_units + < truth.record.required_per_store_dig_base_units, + "the forgery must under-post, which is the stated threat direction" + ); + (truth, lie) + } + + /// GATING finding 1 (dig-node#398 round 2), written as the exploit rather than as an assertion + /// about the code. + /// + /// Seven sybils out of a thousand-strong population return a consistent lie; five honest peers + /// answer truthfully. `agreement_threshold` is a fixed 7 at any population of 20 or more, so + /// counting it as an absolute against a 12-strong responder set adopts a 7/12 PLURALITY — and + /// `SPEC.md` §24.10 says a plurality MUST NOT be adopted. + /// + /// The five honest responders are the control: without a truthful cohort present the sample + /// would fail to converge for an unrelated reason, and the test would pass while proving + /// nothing about the threshold. + #[test] + fn a_seven_strong_plurality_is_not_adopted_when_responders_exceed_the_planned_sample() { + let prior = genesis(); + let (truth, lie) = truth_and_a_consistent_lie(&prior); + + let mut responses = Vec::new(); + for i in 0..7 { + responses.push(PeerRecord { + responder: format!("sybil-{i}"), + record: lie, + }); + } + for i in 0..5 { + responses.push(PeerRecord { + responder: format!("honest-{i}"), + record: truth, + }); + } + + let outcome = adopt(&prior, Some(POPULATION_ABOVE_THE_SAMPLE_CAP), &responses); + assert_eq!( + outcome, + AdoptOutcome::SampleExceeded { + sample_size: 9, + responders: 12, + }, + "7 of 12 is a plurality and must not be adopted" + ); + // Stated separately and deliberately: the point is not which refusal was returned, it is + // that the forged figure did not become this node's history. + assert!( + !matches!(outcome, AdoptOutcome::Adopted { .. }), + "the under-posting record must not be adopted by any route" + ); + } + + /// The bound from BOTH sides, so it cannot be confirmed only from below. + /// + /// At the planned sample of 9 a 7-agreeing cohort is the designed supermajority and is still + /// adopted; one responder over, the same 7 no longer carries. Without the first half a fix + /// that refused every sample would pass. + #[test] + fn seven_of_nine_still_adopts_and_seven_of_ten_does_not() { + let prior = genesis(); + let (truth, lie) = truth_and_a_consistent_lie(&prior); + let at_bound: Vec = (0..7) + .map(|i| PeerRecord { + responder: format!("agreeing-{i}"), + record: lie, + }) + .chain((0..2).map(|i| PeerRecord { + responder: format!("dissenting-{i}"), + record: truth, + })) + .collect(); + assert_eq!(at_bound.len(), 9); + match adopt(&prior, Some(POPULATION_ABOVE_THE_SAMPLE_CAP), &at_bound) { + AdoptOutcome::Adopted { record } => assert_eq!(record.record, lie.record), + other => panic!("7 of the planned 9 is the designed supermajority, got {other:?}"), + } + + let mut one_over = at_bound; + one_over.push(PeerRecord { + responder: "dissenting-2".to_string(), + record: truth, + }); + assert_eq!( + adopt(&prior, Some(POPULATION_ABOVE_THE_SAMPLE_CAP), &one_over), + AdoptOutcome::SampleExceeded { + sample_size: 9, + responders: 10, + } + ); + } + + /// Finding 3: one peer inside an agreeing cohort must not be able to wedge every later epoch. + /// + /// The census height is excluded from the tally key, so agreeing responders can still differ + /// on it. The wedger sorts FIRST by responder id — the order the tally used to take its answer + /// from — and offers `u32::MAX`, past which no later census can ever advance. + #[test] + fn a_hostile_census_height_inside_an_agreeing_cohort_does_not_wedge_later_epochs() { + let prior = genesis(); + let agreed = honest(&prior, 5_625, 40, 9_000); + let mut wedge = agreed; + wedge.census_height = Some(u32::MAX); + assert_eq!( + wedge.record, agreed.record, + "the wedger must AGREE on the consensus record, or it is simply outvoted" + ); + + let mut responses = vec![PeerRecord { + // Sorts before every "honest-*" id, which is what made "first seen" attacker-chosen. + responder: "aaa-wedger".to_string(), + record: wedge, + }]; + for i in 0..6 { + responses.push(PeerRecord { + responder: format!("honest-{i}"), + record: agreed, + }); + } + + match adopt(&prior, Some(PLATEAU_POPULATION), &responses) { + AdoptOutcome::Adopted { record } => { + assert_eq!( + record.census_height, + Some(9_000), + "the honest height must be carried, not the wedger's" + ); + // The property that actually matters: a later epoch can still advance past it. + let next = honest(&record, 6_000, 41, 9_500); + assert_eq!( + verify(&record, &next), + Ok(()), + "a hostile height inside the cohort must not deny every later epoch" + ); + } + other => panic!("the cohort agreed and should adopt, got {other:?}"), + } + } + + /// The coupling between findings 3 and 4, pinned rather than only documented. + /// + /// `None` orders BELOW every `Some` in Rust, so the lowest-height rule alone would let one + /// responder inside an agreeing cohort strip the height and have that carried — leaving an + /// adopted record with no height, which silently disables the NEXT epoch's advancing check. + /// `verify`'s `CensusHeightMissing` refusal is what stops the stripped answer being tallied at + /// all. Removing either fix alone reopens the denial from one end or the other. + #[test] + fn a_stripped_census_height_is_not_carried_by_the_lowest_height_rule() { + let prior = genesis(); + let agreed = honest(&prior, 5_625, 40, 9_000); + let mut stripped = agreed; + stripped.census_height = None; + assert_eq!( + stripped.record, agreed.record, + "the stripper must AGREE on the consensus record, or it is simply outvoted" + ); + + let mut responses = vec![PeerRecord { + // Sorts first, and offers the value that `Ord` ranks lowest of all. + responder: "aaa-stripper".to_string(), + record: stripped, + }]; + for i in 0..7 { + responses.push(PeerRecord { + responder: format!("honest-{i}"), + record: agreed, + }); + } + + match adopt(&prior, Some(PLATEAU_POPULATION), &responses) { + AdoptOutcome::Adopted { record } => { + assert_eq!( + record.census_height, + Some(9_000), + "an absent height must not win the minimum" + ); + // The consequence, not just the field: the next epoch is still checkable. + let next = honest(&record, 6_000, 41, 9_500); + assert_eq!(verify(&record, &next), Ok(())); + } + other => panic!("seven honest responders agreed and should adopt, got {other:?}"), + } + } + + /// Finding 4: omitting the census height must not be a way to opt out of the advancing check. + #[test] + fn a_record_for_a_censused_epoch_with_no_height_is_refused_rather_than_skipped() { + let prior = genesis(); + let mut heightless = honest(&prior, 5_625, 40, 9_000); + heightless.census_height = None; + assert!(heightless.record.epoch > GENESIS_EPOCH); + assert_eq!( + verify(&prior, &heightless), + Err(Rejection::CensusHeightMissing { + epoch: heightless.record.epoch + }) + ); + } +} diff --git a/crates/dig-node-service/src/control.rs b/crates/dig-node-service/src/control.rs index 8347dfe..90281a0 100644 --- a/crates/dig-node-service/src/control.rs +++ b/crates/dig-node-service/src/control.rs @@ -3350,8 +3350,13 @@ fn collateral_margin_set(id: Value, params: &Value) -> Value { Err(e) => return control_error(id, ErrorCode::InvalidParams, e.message), }; + // LOADED, then modified. A struct literal here would name every field, so each preference + // added to the config in future would be silently reset to its default by a caller setting the + // margin — the operator's retention choice erased by an unrelated command, with a success + // reported. Setting one field must change one field. let cfg = crate::collateral::CollateralConfig { margin_bp: parsed.margin_bp, + ..crate::collateral::CollateralConfig::load() }; // Persisted before it is reported. A margin that lapsed to the default on reboot would silently // change what the node posts, so a write failure must not be answered with a success. diff --git a/crates/dig-node-service/src/control_cli.rs b/crates/dig-node-service/src/control_cli.rs index 9ea0941..96c5a66 100644 --- a/crates/dig-node-service/src/control_cli.rs +++ b/crates/dig-node-service/src/control_cli.rs @@ -1227,6 +1227,100 @@ fn compact(result: &Value) -> String { serde_json::to_string(result).unwrap_or_else(|_| "{}".to_string()) } +/// `dign collateral history` — the epochs this node has recorded, and how it came by each. +/// +/// Read from the record file directly rather than over a control call, like `dign spends`: the +/// store is this node's own state on this node's own disk, and an operator diagnosing a node that +/// will not start is exactly the person who most needs to read it. +/// +/// # Provenance is shown, never summarised away +/// +/// A bootstrap record, a censused one and one adopted from a sample of untrusted peers are three +/// different claims about how much this node knows, and the recommendation an operator funds is +/// derived from whichever it holds. Rendering them identically would present the weakest with the +/// authority of the strongest. +pub fn collateral_history(epoch: Option) -> std::io::Result { + use crate::collateral::{EpochRecordStore, StoredEpoch}; + + let store = EpochRecordStore::in_state_dir(); + + // A single epoch goes through `get`, which distinguishes "never recorded" from "recorded and + // unreadable". The listing cannot make that distinction and does not pretend to. + if let Some(epoch) = epoch { + let (human, result) = match store.get(epoch) { + StoredEpoch::Found(record) => (render_record(&record), serde_json::to_value(*record)?), + StoredEpoch::Absent => ( + format!( + "epoch {epoch}: NOT RECORDED — this node has not censused it and has not \ + adopted it from peers." + ), + json!({ "epoch": epoch, "record": Value::Null, "reason": "not_recorded" }), + ), + StoredEpoch::Unreadable => ( + format!( + "epoch {epoch}: RECORDED BUT UNREADABLE — the line for this epoch in {} could \ + not be parsed. The figures are lost, not absent.", + store.path().display() + ), + json!({ "epoch": epoch, "record": Value::Null, "reason": "record_unreadable" }), + ), + }; + return Ok(Outcome::new(human, result)); + } + + let records = store.records()?; + let human = if records.is_empty() { + format!( + "no collateral epochs recorded yet ({}).", + store.path().display() + ) + } else { + let mut lines = Vec::with_capacity(records.len() + 1); + lines.push(format!("{} epoch(s) recorded:", records.len())); + lines.extend(records.iter().map(render_record)); + lines.join("\n") + }; + let result = json!({ + "path": store.path().display().to_string(), + "records": records + .iter() + .map(serde_json::to_value) + .collect::, _>>()?, + }); + Ok(Outcome::new(human, result)) +} + +/// One recorded epoch, as a single operator-readable line. +fn render_record(record: &crate::collateral::StoredRecord) -> String { + use crate::collateral::{format_dig, RecordProvenance}; + + let provenance = match record.provenance { + RecordProvenance::Bootstrap => "genesis (derived from nothing)".to_string(), + RecordProvenance::Censused => "censused by this node".to_string(), + RecordProvenance::AdoptedFromPeers { agreed, sampled } => { + format!("adopted from peers ({agreed} of {sampled} agreed, each re-derived here)") + } + }; + // An absent census height is stated as absent. A "0" here would read as a real block. + let height = match record.census_height { + Some(height) => format!("census height {height}"), + None => "no census (epoch 1 is derived from nothing)".to_string(), + }; + format!( + " epoch {} · {} DIG per store · v{} rules · {} advertisement(s) across {} \ + collateralised owner(s) · multiplier {}.{:06}x · {} · {}", + record.record.epoch, + format_dig(record.record.required_per_store_dig_base_units), + record.record.protocol_version.0, + record.record.census.stores, + record.record.census.owners, + record.record.multiplier_micros / 1_000_000, + record.record.multiplier_micros % 1_000_000, + height, + provenance, + ) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/dig-node-service/src/entrypoint.rs b/crates/dig-node-service/src/entrypoint.rs index c8ce83f..fa649fd 100644 --- a/crates/dig-node-service/src/entrypoint.rs +++ b/crates/dig-node-service/src/entrypoint.rs @@ -611,6 +611,19 @@ enum CollateralCommand { #[arg(long)] balance: Option, }, + /// Show the collateral epochs this node has recorded, and how it came by each. + /// + /// Read from this node's own state directory, so it works whether or not the node is running. + /// Each line says whether the epoch was derived from nothing, censused by this node, or + /// adopted from a sample of peers — three different claims about how much this node knows. + History { + /// Show one epoch instead of the whole history. + /// + /// An epoch this node never recorded is reported as NOT RECORDED, distinctly from one it + /// recorded and can no longer read. + #[arg(long)] + epoch: Option, + }, /// Show your local safety margin, and what it adds. Margin { #[command(subcommand)] @@ -874,6 +887,9 @@ pub fn run() -> std::process::ExitCode { ), Err(e) => emit_error(&e, action, json), }, + Command::Collateral { + action: Some(CollateralCommand::History { epoch }), + } => render(control_cli::collateral_history(epoch), action, json), Command::Collateral { action: cmd } => match collateral_action(cmd) { Ok(a) => render(control_cli::run(&config, a), action, json), Err(e) => emit_error(&e, action, json), @@ -1089,6 +1105,12 @@ fn collateral_action(cmd: Option) -> std::io::Result Ok(ControlAction::CollateralRequirement), // Handled before this mapper: it composes three control reads rather than dispatching one. Some(CollateralCommand::Buffer { .. }) => Ok(ControlAction::CollateralBuffer), + // Also handled before this mapper: it reads this node's own record file directly, so it + // answers on a node that is not running. Mapping it to a control method would make the + // one command an operator reaches for while diagnosing a dead node need a live one. + Some(CollateralCommand::History { .. }) => Err(std::io::Error::other( + "collateral history is served from the local record store, not a control method", + )), Some(CollateralCommand::Margin { action: None }) => Ok(ControlAction::CollateralMarginGet), Some(CollateralCommand::Margin { action: Some(MarginCommand::Set { value }), diff --git a/crates/dig-node-service/src/lib.rs b/crates/dig-node-service/src/lib.rs index c15ce4a..f6d0929 100644 --- a/crates/dig-node-service/src/lib.rs +++ b/crates/dig-node-service/src/lib.rs @@ -36,6 +36,10 @@ pub mod cli; /// safety margin, and the funding advice built on them. Every figure comes out of /// `dig-mirror-collateral`; no formula is restated. pub mod collateral; +/// Adopting an epoch history from untrusted peers, and serving one to them: the +/// re-derivation every candidate record must survive, and the sampling plan that bounds +/// what a sample of peers is allowed to decide. +pub mod collateral_sync; pub mod config; /// Pure HTTP helpers for the local plaintext content-serve surface (#289): `/s/...` route parsing, /// ``/Referer store-root rerooting, the content-type map, the SPA-vs-asset classifier, and the diff --git a/crates/dig-node-service/src/meta.rs b/crates/dig-node-service/src/meta.rs index 3eaeda6..9a585f2 100644 --- a/crates/dig-node-service/src/meta.rs +++ b/crates/dig-node-service/src/meta.rs @@ -210,6 +210,20 @@ pub fn methods() -> &'static [MethodInfo] { loopback-only `GET /health` and the token-gated `control.status`.", requires_auth: false, }, + MethodInfo { + // dig-node#387 — the gossip serve half of the per-epoch collateral record. OPEN, and + // deliberately so: an epoch record is a recomputable consensus value carrying no + // secret, and a peer that cannot fetch history has to re-census it from chain, which + // is the expensive path this method exists to spare the network. A caller verifies + // what it receives by re-derivation regardless of who served it, so authenticating + // the SERVER would buy nothing the verification does not already give. + name: "dig.getCollateralEpoch", + served: "shell", + summary: "This node's record for one mirror-coin collateral epoch, with the census \ + inputs a caller re-derives it from. Params { epoch }; result { record } or \ + { record: null, reason } naming why this node cannot answer. OPEN.", + requires_auth: false, + }, MethodInfo { // #1997: served by the SHELL from the catalogue below — the agent self-describe // (§6.2) that must not depend on an upstream being configured. diff --git a/crates/dig-node-service/src/server.rs b/crates/dig-node-service/src/server.rs index aef2fc2..18155e6 100644 --- a/crates/dig-node-service/src/server.rs +++ b/crates/dig-node-service/src/server.rs @@ -1073,6 +1073,25 @@ async fn rpc( ); } + // `dig.getCollateralEpoch` — the gossip serve half of the per-epoch collateral record + // (dig-node#387). A peer asking about a PAST epoch is answered from this node's own store + // rather than sent off to re-census a week of chain history. + // + // Answered by the shell, like `dig.health`, and for the same reason: a node states what it + // holds on its own authority, and needing an upstream to answer would make a node that has + // recorded an epoch unable to say so. + // + // A record this build cannot interpret is NOT served as a record. It is served as a refusal + // naming the version, so the caller learns the useful fact — this node is behind the ruleset — + // instead of receiving figures neither end can vouch for. + if method == "dig.getCollateralEpoch" { + let result = collateral_epoch_answer(req.get("params").unwrap_or(&Value::Null)); + return ( + StatusCode::OK, + Json(json!({ "jsonrpc": "2.0", "id": id, "result": result })), + ); + } + // PAIRING plane (#280): `pairing.request` / `pairing.poll` are OPEN (no token) — // an MV3 extension can't read the control-token file, so it bootstraps a scoped // credential here. They are NOT under `control.` (so the gate below leaves them @@ -2079,6 +2098,15 @@ where let wallet_backend = state.wallet.clone(); let wallet_cert = state.wallet_cert.clone(); + // Bring the per-epoch collateral record store up (dig-node#387). Two steps, both cheap and + // both synchronous, because a node that answered `dig.getCollateralEpoch` before its own + // genesis record existed would report "not recorded" for an epoch it can derive from nothing. + // + // This is the store's PRODUCTION WRITER. Until it existed the store was written only by its + // own tests, so `control.collateral.requirement` answered `unknown / not_censused` on every + // node in the network — a correct answer, and a permanently unchanging one. + bring_up_collateral_records(); + // §14 autonomous sync (#213): bring up the L7 peer network — the connected peer // pool, the content-location DHT + P2P content engine, PEX, and the chain-watch + // generation gap-fill loop — so a running node tracks the chain and PROACTIVELY @@ -2421,6 +2449,120 @@ async fn shutdown_signal() { tracing::info!("dig-node shutting down"); } +/// Answer `dig.getCollateralEpoch` from this node's own record store. +/// +/// # Every non-answer is a NAMED refusal, never a shaped-like-success zero +/// +/// The four ways this can fail to produce a record — an unreadable epoch parameter, an epoch never +/// recorded, a line that cannot be read, and a record governed by a ruleset this build does not +/// implement — are four different facts with four different remedies, and each is returned as +/// `record: null` beside its own `reason` token. This node's own read path pays for that +/// distinction already (`crate::collateral::StoredEpoch`), and discarding it at the wire boundary +/// would hand every caller the same unactionable sentence. +/// +/// It matters more here than locally, because the caller is a peer deciding whether to re-census a +/// week of chain history. "I never recorded it" means ask someone else; "I cannot read my copy" +/// means this node is broken and should not be asked again; "your ruleset is newer than mine" +/// means the fault is not the caller's at all. +/// +/// # Why the read is unauthenticated, and what bounds it +/// +/// An epoch record is a recomputable consensus value carrying nothing secret, and a caller +/// verifies it by re-derivation whatever its source — so authenticating the server would buy +/// nothing. The cost is one read of the record file per request. That file holds one line per +/// epoch and an epoch is a week, so it is on the order of tens of lines even under the default +/// keep-everything retention; it is bounded by the calendar rather than by anything a caller +/// controls. +fn collateral_epoch_answer(params: &Value) -> Value { + use crate::collateral::{EpochRecordStore, StoredEpoch}; + + // Read typed and refuse what will not decode. A `params.epoch` that is absent, negative, or + // not a number must NOT fall through to a default — epoch 0 is not an epoch, and epoch 1 is a + // real record that a defaulting reader would serve in answer to a question nobody asked. + let Some(epoch) = params + .get("epoch") + .and_then(Value::as_u64) + .filter(|e| *e >= 1) + else { + return json!({ + "record": Value::Null, + "reason": "invalid_epoch", + "detail": "params.epoch is required and is a one-based epoch number", + }); + }; + + match EpochRecordStore::in_state_dir().get(epoch) { + // The protocol-version ceiling, applied on the SERVE side too. A record this build cannot + // interpret is one it cannot vouch for, and passing it on unremarked would launder an + // unverifiable record through a node that never checked it. + StoredEpoch::Found(record) if !record.is_interpretable() => json!({ + "record": Value::Null, + "reason": "unimplemented_ruleset", + "protocol_version": record.record.protocol_version.0, + }), + StoredEpoch::Found(record) => match serde_json::to_value(*record) { + Ok(record) => json!({ "record": record }), + Err(e) => json!({ + "record": Value::Null, + "reason": "record_unreadable", + "detail": e.to_string(), + }), + }, + StoredEpoch::Absent => json!({ "record": Value::Null, "reason": "not_recorded" }), + StoredEpoch::Unreadable => json!({ "record": Value::Null, "reason": "record_unreadable" }), + } +} + +/// Seed the collateral record store and apply the operator's retention preference. +/// +/// Best-effort and never fatal: a node that cannot write its record store still serves content, +/// and taking the whole node down over it would trade a missing figure for an outage. Every +/// failure is logged at WARN rather than swallowed, because the observable symptom otherwise is +/// `control.collateral.requirement` answering `unknown` forever with no stated cause. +fn bring_up_collateral_records() { + use crate::collateral::{ + current_epoch_now, ensure_bootstrap, CollateralConfig, CurrentEpoch, EpochRecordStore, + PutOutcome, + }; + + let store = EpochRecordStore::in_state_dir(); + match ensure_bootstrap(&store) { + Ok(PutOutcome::Written) => { + tracing::info!(path = %store.path().display(), "recorded the genesis collateral epoch") + } + Ok(PutOutcome::AlreadyPresent) => {} + // A genesis record that disagrees with this build's own `EpochRecord::bootstrap` is not a + // write to retry — it means the node's history was written under different rules, or was + // tampered with. It is kept and reported; overwriting it is the one thing that must not + // happen quietly. + Ok(PutOutcome::Conflict { held }) => tracing::warn!( + path = %store.path().display(), + held_protocol_version = held.record.protocol_version.0, + "the stored genesis collateral epoch differs from this build's; keeping the stored one" + ), + Err(e) => tracing::warn!( + path = %store.path().display(), + error = %e, + "could not record the genesis collateral epoch" + ), + } + + // Retention. `KeepEverything` — the default — reads nothing and writes nothing, so a node that + // never opted in never rewrites this file at all. + let policy = CollateralConfig::load().retention(); + if let CurrentEpoch::Final(epoch) = current_epoch_now() { + match store.prune(policy, epoch) { + Ok(0) => {} + Ok(dropped) => tracing::info!( + dropped, + policy = ?policy, + "truncated the collateral record history at the operator's configured retention" + ), + Err(e) => tracing::warn!(error = %e, "could not apply the collateral retention policy"), + } + } +} + #[cfg(test)] mod tests { use super::{