Skip to content

Add integer max-log (frontier_lite) metric mode to the trellis engine - #589

Merged
ciaranra merged 5 commits into
devfrom
frontier-maxlog-metric
Aug 30, 2026
Merged

Add integer max-log (frontier_lite) metric mode to the trellis engine#589
ciaranra merged 5 commits into
devfrom
frontier-maxlog-metric

Conversation

@ciaranra

Copy link
Copy Markdown
Member

Summary

Stacked on #587; retarget to dev when that merges. Adds upstream's frontier_lite / maxlog_int metric mode: the DP runs on scaled int64 masses with max-log (Viterbi) semantics, for both the binary and N-ary kernels.

  • TrellisConfig gains metric_mode (LogSumExpFloat default, MaxLogInt) and int_metric_scale (default 1024), named as upstream names them.
  • Faithful fixed-point port: saturating quantization with round-half-away-from-zero, the scale-1024 shift fast path, NegInf propagation, per-column constants quantized once at build. Candidate merge and the terminal per-label fold take max; the winner ties toward the smaller label, matching upstream.
  • Under MaxLogInt, log_evidence is the winning route mass (not a log-sum), logical_masses are per-label best-route masses, and runner_up_gap is a route-mass margin; the result rustdoc and both decoder docs say so explicitly (Rust API only for now).
  • One deliberate deviation from upstream, verified live before porting: upstream quantizes Delta=inf to the negative sentinel and clamps it to zero, silently turning "no delta pruning" into "prune to score-ties" — it changed a decoded label in a direct probe. PECOS rejects non-finite delta under MaxLogInt at construction instead.
  • FrontierCommittee rejects MaxLogInt: its arbitration ranks on coset-mass posteriors, and one ranking component degenerates to a constant under max-log. BP-Trellis pins the float metric internally (its config exposes no metric knob).
  • Float paths are behavior-frozen: suffix tables only compute quantized fields when the integer metric is selected (each table row carries a scale tag asserted before integer scoring), and the byte-frozen binary fixtures plus the N-ary oracle fixtures pass unmodified.

Verification

  • New frozen oracle fixtures (upstream_maxlog_fixtures.json): 5 binary models decoded by upstream's native binary engine under frontier_lite at scales 1024 and 64, wide and pruned; PECOS matches status, predicted label, every terminal mass, log_evidence, and the top-mass gap within 1e-9. The maxlog_metric_flips_winner fixture proves the metric actually switched: the same model and syndrome decode to label 0 under the float metric and label 1 under max-log.
  • N-ary + max-log has no upstream oracle (upstream restricts the mode to its binary engine): a brute-force enumeration oracle with independently hardcoded quantized constants asserts exact integer equality.
  • Quantizer and fixed-point unit tests at the saturation boundaries, exact-half rounding both signs (±2.5/scale, the case that separates half-away from ties-to-even), and fast-path/generic-path agreement.
  • An independent adversarial review ran against the diff and the upstream C++; every finding was verified and closed, including a measured ~25% per-shot regression on the default float BP path (int fields computed but never read — now gated), the committee guard above, and six mutation-surviving test gaps (all six mutants now killed, plus a re-run spot check).
  • cargo test on the three engine crates (16 suites), cold clippy --all-targets -- -D warnings including pecos-rslib-exp, cargo fmt --check, and the full pre-commit lane all pass.

Follow-up (separate PR): Python bindings expose the metric knob (Phase C).

@ciaranra

Copy link
Copy Markdown
Member Author

Review-panel follow-up pushed, two execution-confirmed defects fixed: (1) score_alpha=0 under maxlog_int no longer evaluates suffix scores, so NegInf-quantized suffix rows from ~1e-17-probability mechanisms can no longer poison and prune feasible states (repro test added; previously flipped the decoded label); (2) merge_indistinguishable is now rejected under maxlog_int — the XOR merge sums coset mass, which silently changed 'Exact' Viterbi answers (verified on the M1 fixture model). Also: integer prune ties now match upstream's order (score desc, log-mass desc, key asc; integer ties are not measure-zero), the suffix-table scale check moved out of the innermost scoring loop, and a +inf quantizer debug assertion was added. All guards mutation-tested; fixtures unchanged.

@ciaranra
ciaranra force-pushed the frontier-maxlog-metric branch from 94c0c1d to 21820ad Compare August 28, 2026 22:42
@ciaranra

Copy link
Copy Markdown
Member Author

Post-panel delta review fixes: the release-mode quantizer test is now cfg-gated (it asserted a debug-only panic and failed under the repo's release-mode just test/check-all gates, while CI's dev-profile lane stayed green) with the release fallback pinned explicitly; a nonzero score_alpha that quantizes to zero at the configured scale is now rejected loudly instead of silently disabling suffix scoring (e.g. score_alpha=0.4 at int_metric_scale=1); and the alpha short-circuit carries a why-comment so it cannot be 'simplified' back into the sentinel-poisoning bug.

@ciaranra
ciaranra force-pushed the frontier-maxlog-metric branch from 21820ad to cc73cf1 Compare August 29, 2026 15:19
Base automatically changed from frontier-nary-factor-model to dev August 30, 2026 22:29
@ciaranra
ciaranra force-pushed the frontier-maxlog-metric branch from cc73cf1 to 9ccb2fc Compare August 30, 2026 22:34
@ciaranra
ciaranra merged commit eebbc77 into dev Aug 30, 2026
33 checks passed
@ciaranra
ciaranra deleted the frontier-maxlog-metric branch August 30, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant