Skip to content

Add multi-outcome factor-model input to the trellis engine - #587

Open
ciaranra wants to merge 8 commits into
devfrom
frontier-nary-factor-model
Open

Add multi-outcome factor-model input to the trellis engine#587
ciaranra wants to merge 8 commits into
devfrom
frontier-nary-factor-model

Conversation

@ciaranra

Copy link
Copy Markdown
Member

Summary

Adds an N-ary (multi-outcome) factor-model input to exp/pecos-trellis, closing the largest capability gap versus the upstream Frontier decoder's choice engine: a factor with arbitrarily many mutually exclusive outcomes, each carrying its own probability, detector toggles, and observable toggles. This also provides the public model-construction API (FactorModel / Factor / Outcome) that previously existed only upstream.

  • FactorModel::new validates outcomes (probabilities in range and summing to 1 within 1e-10, indices in range, no duplicates per outcome); TryFrom<&SparseDem> maps each binary mechanism to a two-outcome factor.
  • TrellisDecoder::from_factor_model classifies the model: binary-shaped models delegate to from_sparse_dem and are bitwise-identical to the equivalent DEM build (BP scoring and indistinguishable-merging fully supported); genuinely multi-outcome models run a new N-ary DP kernel.
  • The N-ary kernel mirrors the binary DP (same merge, prune, ordering, and forced-layer semantics) with per-outcome log-prior branches, and generalizes suffix-compatibility scoring to per-row toggle probabilities.
  • A two-outcome factor delegates only when its stored baseline agrees with the computed complement to within 1e-12 relative — float-roundoff-level agreement in either listing order; larger disagreement is deliberate parameterization and is evaluated faithfully on the N-ary kernel with each outcome's stored probability.
  • Binary-only features fail loud on genuinely N-ary models: BP-guided pruning is rejected whenever it would engage (inert on the unpruned fast path, matching the binary kernel), and indistinguishable-mechanism merging is rejected unconditionally.
  • deadline_column_order_for_factors / backward_deadline_column_order_for_factors order factors by the union of their outcomes' detector supports.

Binary path unchanged

The binary kernel is under a byte-identical external parity contract. Every existing fixture (upstream_fixtures.json, upstream_order_fixtures.json, bitwise_snapshot.json) passes unmodified; the refactors on the shared path (kernel enum wrapping, mask-slice plumbing in merge_branch, integer-only ordering refactor) move zero float operations, verified by an independent mechanical diff of the decode functions.

Verification

  • New frozen oracle fixtures (upstream_nary_fixtures.json): 7 N-ary models decoded per-shot against the upstream choice engine, cross-checked against the upstream pure-Python reference on unpruned decodes, hand-verified on the degeneracy case. PECOS matches status, predicted label, every terminal log mass, and log evidence within 1e-9, unpruned and pruned (pruned rows also pin the retained-label set, validating the generalized suffix scoring).
  • Brute-force enumeration oracle over seeded random N-ary models, all syndromes, exact-decode regime.
  • Equivalence: two-outcome factors with nonempty baselines vs their forced-toggle binary DEM image — identical decisions, masses within 1e-9.
  • Delegation: DEM-derived and hand-written decimal-literal factor models decode bitwise-identically to the DEM build, in both outcome listing orders, including with BP scoring and merging enabled.
  • Every new guard (probability sum, empty factor, index validation, delegation gate both directions, BP and merge rejection, column-order permutation) is mutation-tested.
  • cargo test -p pecos-trellis -p pecos-frontier -p pecos-bp-trellis, cold cargo clippy --all-targets -- -D warnings, and cargo fmt --check all pass.

Follow-ups (separate PRs): integer max-log metric mode, Python bindings and user-facing docs.

@ciaranra

Copy link
Copy Markdown
Member Author

Review-panel follow-up pushed: the binary-delegation gate now accepts pairs whose induced baseline log-mass error is within the engine's 1e-9 acceptance bar (the previous 1e-12-relative bound was unreachable for small baselines, demoting exact-complement factors like (1e-6, 0.999999) off the binary kernel). New tests: the repro in both listing orders (bitwise vs the DEM build, BP enabled), a tiny-baseline case documenting why sub-2e-7 baselines correctly stay on the faithful N-ary kernel, and a seeded classification sweep. Gate mutation-tested in both directions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant