Skip to content

Split the test runner out of Hardened.Shared.Testing - #273

Merged
ipjohnson merged 4 commits into
mainfrom
runner-split
Sep 5, 2026
Merged

ipjohnson merged 4 commits into
mainfrom
runner-split

Conversation

@ipjohnson

Copy link
Copy Markdown
Owner

What

Hardened.Shared.Testing becomes runner-neutral, and two runner packages sit beside it:

  • Hardened.Shared.Testing.xUnit: [HardenedTest] as it was, the logger that writes to a test's output, and the xUnit half of the new running-test seam. Every xUnit test project references it beside the core; the three templates do too.
  • Hardened.Shared.Testing.NUnit: [HardenedTest] under the same name and namespace over DependencyModules.NUnit, a logger writing to NUnit's TestContext.Out, and the NUnit half of the seam.

The seam is CurrentTest in the core: Key (one object per running test, alive as long as it, flowing through async code), Assembly, DisplayName, and the runner's logger provider, installed by the static constructor of each package's [HardenedTest], which the runner instantiates while it discovers tests. LastResponse, Kiota's recording handler and Returns<T>() read it instead of xUnit's TestContext, and Hardened.Web.Testing and Hardened.Kiota.Testing drop their xunit reference. WebAssertThat throws the harness's own WebAssertionException (Expected status 404, the response was 500.) instead of Xunit.Assert.

Why

TEST-HOST-PLAN.md adds test hosts that bind a loopback socket per test, and the ask was that they work under xUnit and NUnit alike. The host seam is runner-neutral by construction, because it only uses the DependencyModules hooks, which DependencyModules.NUnit calls in the same order. The harness around it was xUnit-bound in six places: the attribute, the logger, LastResponse, the Kiota recorder, the assembly lookup behind Returns<T>(), and Xunit.Assert in the status assertions. A neutral web harness cannot sit on an xUnit-bound core without dragging xUnit into every NUnit project, and with the core xUnit-bound the NUnit [HardenedTest] would have needed another name. So the split.

Held by

  • Hardened.Shared.Testing.NUnit.Tests, the first NUnit project in the repository: a parameter from the entry point's container, a [Mock] beating the application's registration, the environment the assembly declares, the key stable across an await and null outside a test, NUnit's logger provider registered, and two tests seeing two containers with the first disposed by the time the second runs.
  • Hardened.IntegrationTests.WebApp.SUT.NUnitTests: the web harness under NUnit, one twin per shape the xUnit project asserts. ITestWebApp, a mock behind a route, a Kiota client and a Refit interface as parameters read by Returns<T>(), LastResponse, two parameters with two credentials, and the harness's own assertion failing by name.
  • The existing suites, unchanged in behaviour: the whole solution passes in Release with the CI flags, and the public API approvals for the four assemblies are updated.

Notes

  • Hardened.Requests.Testing keeps its xunit reference for the conformance suites, which are xUnit test classes by design; an NUnit project referencing the web harness sees xunit assemblies it never uses. Splitting the conformance suites out is the follow-up if that ever matters.
  • A module initializer in each runner package was the first shape and is not used: CA2255 refuses one in a library, and it added nothing the static constructor does not do. A test project that drives the harness directly and declares no [HardenedTest] installs the seam itself; the three harness test projects do.
  • Pack list 25 to 27. The two new assemblies are not in coverage-baseline.json, which the gate reads as unmeasured rather than as a failure; a floor from CI's report can follow.
  • Amz's next bump adds Hardened.Shared.Testing.xUnit to its two test projects that use [HardenedTest] (SqsTest.Tests, DynamoDbStreamApp.Tests); no shipped Amz package touches what moved.
  • The assembly-end leak test from the plan's section 3.6 needs DependencyModules 1.3.1 (Dispose a test's container when its case has run DependencyModules#56) and comes with that bump, separately.
  • docs/testing-conventions.md section 12 says which package a test project references.

🤖 Generated with Claude Code

ipjohnson and others added 4 commits September 5, 2026 08:38
The harness was xUnit-bound in six places: [HardenedTest], the logger,
LastResponse, Kiota's recording handler, the assembly lookup behind
Returns<T>(), and Xunit.Assert in the status assertions. The core now
names no runner; Hardened.Shared.Testing.xUnit and
Hardened.Shared.Testing.NUnit each ship [HardenedTest] and fill the
CurrentTest seam the harness reads, so a test project references the
core and one of the two, and a [HardenedTest] reads the same under
either runner.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The coverage gate reported Hardened.Shared.Testing below its floor once
the xUnit logger and attribute moved out: what stayed gained CurrentTest
and the entry point's no-runner branch, and nothing exercised them. The
tests also found that a plain [Fact] can run before any [HardenedTest]
has been read, so the seam is installed by the tests that read it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two of them replace the process-wide provider, and in CI another
class's Install() landed inside the window one had opened. A collection
with parallelization disabled runs alone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hardened.Shared.Testing's line floor drops from 97.0 to 96.3, CI's own
measurement on #273: the xUnit logger and attribute, covered by every
test that ran, moved to Hardened.Shared.Testing.xUnit, and what stayed
kept its uncovered lines. The two runner packages get floors from the
same report. NUnit test assemblies are excluded from the report the way
the xUnit ones are, by name.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant