ci: add per-crate feature matrix check with cargo-hack - #241
Conversation
Both existing Rust checks run workspace-scoped, so cargo unifies features across members and each crate's feature set is only ever validated in combination — never in the configuration a downstream consumer or `cargo publish` would build. Two shipped bugs came from that blind spot (circlefin#233, circlefin#236), and neither existing job reports them: cargo check --workspace --all-features # 0 errors cargo check --workspace --no-default-features # 0 errors cargo-hack decomposes `--workspace` into per-package runs (`cargo check --manifest-path crates/<x>/Cargo.toml ...`), which is the whole mechanism — adding flags to the workspace-scoped jobs catches nothing. `arc-signer` gets its own invocation. Its `compile_error!("At least one signing provider feature must be enabled")` guard is correct code, but cargo-hack counts the bare `--no-default-features` run as a failure. `--exclude-no-default-features` drops exactly that one configuration and still checks each of the crate's five features individually, rather than exempting the crate wholesale. Closes circlefin#240 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
osr21
left a comment
There was a problem hiding this comment.
Reviewed against #240 (which I filed) and against the repo's existing CI conventions — approving, and all three deviations from the issue's proposed config are improvements, not compromises:
1. --exclude-no-default-features is the right correction to my config. You're right that --at-least-one-of is powerset-only — that constraint is in cargo-hack's own docs and I proposed an invalid flag combination; thanks for catching it by running rather than trusting the issue. The substitute preserves exactly the property I was defending (every one of arc-signer's 5 features still individually checked, only the guard-forbidden empty set dropped), and your 6-configuration listing proves it. On the powerset offer: I'd say no for now — 2^5 runs for one crate buys combination coverage nobody has yet demonstrated a bug class for, while --each-feature has two shipped bugs as its fixture. Powerset can be a follow-up the day a combination bug actually appears.
2. Convention fidelity verified line-by-line. Checkout pinned to the same df4cb1c0 SHA as every other job, same apt deps as the build/test jobs, same toolchain action, same taiki-e/install-action@v2 the nextest job uses, --locked throughout, needs: rust-lint consistent with rust-test's gating. The separate rust-feature-matrix cache key is the right default — 63 check permutations would churn rust-build's cache for the jobs that compile full artifacts; reuse can be revisited if cache storage becomes the constraint, but eviction pressure on the build jobs is the more expensive failure.
3. Dropping --no-dev-deps is correct and the reasoning deserves to be on record: it's not that dev-deps coverage isn't wanted, it's that cargo-hack's manifest-rewriting implementation is mechanically incompatible with --locked, and diverging from the repo-wide --locked convention silently would be worse than the marginal coverage. If publishing-without-dev-deps correctness ever matters (it will, the day these crates actually publish), that's a cargo publish --dry-run job, not a flag on this one.
For maintainers, one sequencing note: this PR's CI is expected red until #231 and #237 merge — that's the acceptance criterion from #240 working as designed (red on the two open bugs, green after their fixes), not a defect. Cleanest order: #231 → #237 (one-line rebase of the [features] table, resolution already written out in this PR's description) → this. The failure list on this PR's first run — exactly configurations 3/57, 4/57, 5/57, nothing else — is the regression fixture proving the job detects precisely the bug class it was built for.
The measured-not-inferred discipline throughout (every claim in the description carries its command and output) made this the easiest CI review I've done on this repo. LGTM pending the two fix PRs landing first.
Closes #240.
Adds the per-crate feature matrix job. Everything below is measured against this branch, not inferred.
What runs
57 + 6 = 63 configurations. The one configuration dropped versus a bare
--each-feature --workspace(64) isarc-signer --no-default-features, which itscompile_error!guard forbids by design.Three deviations from the config proposed in #240
1.
--at-least-one-of local,remotedoes not work with--each-feature. cargo-hack rejects it outright:--exclude-no-default-featuresreaches the same goal on the--each-featurepath: it drops only the bare--no-default-featuresrun and still checks all five features individually, so the gaps you raised stay covered —integrationalone andlocal/remoteseparately are all exercised. Switching to--feature-powersetwould also work and would additionally cover combinations, at 2^5 runs for this crate — happy to do that instead if you want the stronger guarantee.2. Toolchain and cache actions follow the repo's existing jobs.
ci.ymlusesactions-rust-lang/setup-rust-toolchain@v1rather thandtolnay/rust-toolchain@stable, and that action wraps rust-cache already viacache-shared-key, so the separateSwatinem/rust-cache@v2step is redundant here.taiki-e/install-action@v2is the same action the nextest job already uses. I gave this job its own cache key (rust-feature-matrix) instead of the sharedrust-build, since 63 permutations would otherwise churn the cache the other Rust jobs depend on — easy to switch to the shared key if you would rather have the reuse.3.
--no-dev-depsis omitted. It was raised on #236, but it is incompatible with--locked: cargo-hack rewrites the manifest to strip dev-dependencies, and cargo then refuses to touch the lockfile.All five configurations fail for that reason rather than on their merits, so the job would be uniformly and misleadingly red. Dropping
--lockedwould make it usable, but every other Rust job here passes--lockedand I would rather not diverge silently.Acceptance criteria, both directions verified
Red on
main— the first command fails on exactly the three configurations the two open bugs predict, and nothing else across the other 54:Green with the fixes — applying #231 and #237 together and re-running:
So this PR's own CI run is expected to fail until #231 and #237 merge. That is the acceptance criterion from #240 rather than a broken PR, and the failure output is the regression fixture.
Merge note
Applying #231 and #237 together conflicts in the
[features]table ofcrates/types/Cargo.toml— the adjacent-line collision flagged on #237. The resolution is one line:with
defaultandsigner-localboth gone. I will rebase whichever of the two lands second.Cost
The earlier 64-configuration run took ~69 min cold-cache locally; this job's 63 are the same work minus one. Worth one warm CI run before deciding whether it belongs on every PR or on merge-queue/nightly — only 8 of 25 crates declare features, so scoping to those is the other lever. I did not pre-empt that decision here since the measurement should drive it.