Skip to content

ci: add per-crate feature matrix check with cargo-hack - #241

Open
mehmetkr-31 wants to merge 1 commit into
circlefin:mainfrom
mehmetkr-31:ci/feature-matrix-cargo-hack
Open

ci: add per-crate feature matrix check with cargo-hack#241
mehmetkr-31 wants to merge 1 commit into
circlefin:mainfrom
mehmetkr-31:ci/feature-matrix-cargo-hack

Conversation

@mehmetkr-31

Copy link
Copy Markdown

Closes #240.

Adds the per-crate feature matrix job. Everything below is measured against this branch, not inferred.

What runs

- run: cargo hack check --each-feature --workspace --exclude arc-signer --locked
- run: cargo hack check --each-feature --exclude-no-default-features -p arc-signer --locked

57 + 6 = 63 configurations. The one configuration dropped versus a bare --each-feature --workspace (64) is arc-signer --no-default-features, which its compile_error! guard forbids by design.

Three deviations from the config proposed in #240

1. --at-least-one-of local,remote does not work with --each-feature. cargo-hack rejects it outright:

$ cargo hack check --each-feature -p arc-signer --at-least-one-of local,remote
error: --at-least-one-of can only be used together with --feature-powerset

--exclude-no-default-features reaches the same goal on the --each-feature path: it drops only the bare --no-default-features run and still checks all five features individually, so the gaps you raised stay covered —

cargo check --all-features                                    on arc-signer (1/6)
cargo check --no-default-features --features default          on arc-signer (2/6)
cargo check --no-default-features --features integration      on arc-signer (3/6)
cargo check --no-default-features --features integration-remote-signer (4/6)
cargo check --no-default-features --features local            on arc-signer (5/6)
cargo check --no-default-features --features remote           on arc-signer (6/6)

integration alone and local/remote separately are all exercised. Switching to --feature-powerset would also work and would additionally cover combinations, at 2^5 runs for this crate — happy to do that instead if you want the stronger guarantee.

2. Toolchain and cache actions follow the repo's existing jobs. ci.yml uses actions-rust-lang/setup-rust-toolchain@v1 rather than dtolnay/rust-toolchain@stable, and that action wraps rust-cache already via cache-shared-key, so the separate Swatinem/rust-cache@v2 step is redundant here. taiki-e/install-action@v2 is the same action the nextest job already uses. I gave this job its own cache key (rust-feature-matrix) instead of the shared rust-build, since 63 permutations would otherwise churn the cache the other Rust jobs depend on — easy to switch to the shared key if you would rather have the reuse.

3. --no-dev-deps is omitted. It was raised on #236, but it is incompatible with --locked: cargo-hack rewrites the manifest to strip dev-dependencies, and cargo then refuses to touch the lockfile.

$ cargo hack check --each-feature --no-dev-deps -p arc-consensus-types --locked
error: cannot update the lock file ... because --locked was passed to prevent this
error: failed to run 5 commands

All five configurations fail for that reason rather than on their merits, so the job would be uniformly and misleadingly red. Dropping --locked would make it usable, but every other Rust job here passes --locked and I would rather not diverge silently.

Acceptance criteria, both directions verified

Red on main — the first command fails on exactly the three configurations the two open bugs predict, and nothing else across the other 54:

cargo check --all-features                            on arc-consensus-types (3/57)   → #233
cargo check --no-default-features                     on arc-consensus-types (4/57)   → #236
cargo check --no-default-features --features arbitrary on arc-consensus-types (5/57)  → both

Green with the fixes — applying #231 and #237 together and re-running:

$ cargo hack check --each-feature -p arc-consensus-types --locked
exit 0, 0 errors

So this PR's own CI run is expected to fail until #231 and #237 merge. That is the acceptance criterion from #240 rather than a broken PR, and the failure output is the regression fixture.

Merge note

Applying #231 and #237 together conflicts in the [features] table of crates/types/Cargo.toml — the adjacent-line collision flagged on #237. The resolution is one line:

[features]
arbitrary = ["dep:arbitrary", "alloy-primitives/arbitrary"]

with default and signer-local both gone. I will rebase whichever of the two lands second.

Cost

The earlier 64-configuration run took ~69 min cold-cache locally; this job's 63 are the same work minus one. Worth one warm CI run before deciding whether it belongs on every PR or on merge-queue/nightly — only 8 of 25 crates declare features, so scoping to those is the other lever. I did not pre-empt that decision here since the measurement should drive it.

Both existing Rust checks run workspace-scoped, so cargo unifies features
across members and each crate's feature set is only ever validated in
combination — never in the configuration a downstream consumer or
`cargo publish` would build. Two shipped bugs came from that blind spot
(circlefin#233, circlefin#236), and neither existing job reports them:

    cargo check --workspace --all-features         # 0 errors
    cargo check --workspace --no-default-features  # 0 errors

cargo-hack decomposes `--workspace` into per-package runs
(`cargo check --manifest-path crates/<x>/Cargo.toml ...`), which is the
whole mechanism — adding flags to the workspace-scoped jobs catches
nothing.

`arc-signer` gets its own invocation. Its
`compile_error!("At least one signing provider feature must be enabled")`
guard is correct code, but cargo-hack counts the bare
`--no-default-features` run as a failure. `--exclude-no-default-features`
drops exactly that one configuration and still checks each of the crate's
five features individually, rather than exempting the crate wholesale.

Closes circlefin#240

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@osr21 osr21 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against #240 (which I filed) and against the repo's existing CI conventions — approving, and all three deviations from the issue's proposed config are improvements, not compromises:

1. --exclude-no-default-features is the right correction to my config. You're right that --at-least-one-of is powerset-only — that constraint is in cargo-hack's own docs and I proposed an invalid flag combination; thanks for catching it by running rather than trusting the issue. The substitute preserves exactly the property I was defending (every one of arc-signer's 5 features still individually checked, only the guard-forbidden empty set dropped), and your 6-configuration listing proves it. On the powerset offer: I'd say no for now — 2^5 runs for one crate buys combination coverage nobody has yet demonstrated a bug class for, while --each-feature has two shipped bugs as its fixture. Powerset can be a follow-up the day a combination bug actually appears.

2. Convention fidelity verified line-by-line. Checkout pinned to the same df4cb1c0 SHA as every other job, same apt deps as the build/test jobs, same toolchain action, same taiki-e/install-action@v2 the nextest job uses, --locked throughout, needs: rust-lint consistent with rust-test's gating. The separate rust-feature-matrix cache key is the right default — 63 check permutations would churn rust-build's cache for the jobs that compile full artifacts; reuse can be revisited if cache storage becomes the constraint, but eviction pressure on the build jobs is the more expensive failure.

3. Dropping --no-dev-deps is correct and the reasoning deserves to be on record: it's not that dev-deps coverage isn't wanted, it's that cargo-hack's manifest-rewriting implementation is mechanically incompatible with --locked, and diverging from the repo-wide --locked convention silently would be worse than the marginal coverage. If publishing-without-dev-deps correctness ever matters (it will, the day these crates actually publish), that's a cargo publish --dry-run job, not a flag on this one.

For maintainers, one sequencing note: this PR's CI is expected red until #231 and #237 merge — that's the acceptance criterion from #240 working as designed (red on the two open bugs, green after their fixes), not a defect. Cleanest order: #231#237 (one-line rebase of the [features] table, resolution already written out in this PR's description) → this. The failure list on this PR's first run — exactly configurations 3/57, 4/57, 5/57, nothing else — is the regression fixture proving the job detects precisely the bug class it was built for.

The measured-not-inferred discipline throughout (every claim in the description carries its command and output) made this the easiest CI review I've done on this repo. LGTM pending the two fix PRs landing first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: per-crate feature matrix check (cargo hack --each-feature) to close the workspace-unification blind spot

2 participants