Skip to content

Add node-local snapshot inspection to test infrastructure - #8317

Merged
Amaury Chamayou (achamayou) merged 9 commits into
microsoft:mainfrom
cjen1-msft:snapshot-infra-01-disk-helpers
Sep 9, 2026
Merged

Amaury Chamayou (achamayou) merged 9 commits into
microsoft:mainfrom
cjen1-msft:snapshot-infra-01-disk-helpers

Conversation

@cjen1-msft

@cjen1-msft cjen1-msft commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Network.get_committed_snapshots() combines snapshot generation, waiting, and copying into common_dir. Tests that only inspect files or wait for a snapshot do not need those copies: the nodes' directories are already accessible locally.

Add two helpers on Node:

  • get_snapshots() returns committed file paths sorted by snapshot seqno. Read-only startup snapshots require include_read_only=True.
  • wait_for_snapshot(target_seqno) waits in that node's writable directory for a committed snapshot whose state seqno covers the target.

Neither helper sends requests, generates transactions, or copies files. Returned paths remain node-owned; callers needing durable or mutable copies must copy them explicitly.

Migrate readers in three files:

  • e2e_operations.py: election and time-based snapshot waits, plus per-node persistence inventory. Callers sample the commit target explicitly, and counts still deduplicate by filename.
  • governance_history.py: read the snapshot covering the target from the existing ledger-chunk setup, rather than the arbitrary last entry from os.listdir(). This removes the old helper's additional ack transactions and relies on normal signature progress to commit the snapshot.
  • recovery_snapshot_endorsements.py: compare snapshot bytes in place, including read-only inputs. If a filename exists in both directories, check both copies rather than merging them.

First part of the replacement for #7783. Explicit generation, recovery, joins, and corruption/LTS migrations follow separately; the old helper remains for those callers.

scripts/ci-checks.sh passes. The targeted timing and persistence runs were blocked at startup by the missing logging executable. The migrated end-to-end scenarios still need execution, particularly the governance reader's passive snapshot wait.

Chris Jensen (Cjen1) and others added 4 commits September 8, 2026 15:32
Add node-local committed snapshot listing and passive target-seqno waiting. Migrate timing, election, and persistence callers without changing generation or recovery-copy behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the helper-only snapshot file tests and their CMake registration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the existing snapshot trigger's target when reading governance history. Compare recovery snapshot files in place, including read-only inputs, without copying into common_dir.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cjen1-msft cjen1-msft changed the title Separate snapshot inspection from snapshot staging Add node-local snapshot inspection to test infrastructure Sep 8, 2026
@cjen1-msft
cjen1-msft requested a balanced review from Copilot September 8, 2026 16:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The migrated end-to-end scenarios remain unexecuted, including the passive governance snapshot wait.

Pull request overview

Adds node-local snapshot inspection and waiting helpers, migrating tests away from unnecessary copying and transaction generation.

Changes:

  • Adds sorted snapshot listing and passive waiting to Node.
  • Updates governance, recovery, timing, and persistence tests.
  • Preserves checks across writable and read-only snapshot directories.
File summaries
File Description
tests/recovery_snapshot_endorsements.py Verifies snapshot bytes in writable and read-only directories.
tests/infra/node.py Adds node-local snapshot listing and waiting helpers.
tests/governance_history.py Selects the snapshot covering the target sequence number.
tests/e2e_operations.py Migrates snapshot timing, election, and persistence checks.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@cjen1-msft
cjen1-msft marked this pull request as ready for review September 9, 2026 08:58
@cjen1-msft
cjen1-msft requested a review from a team as a code owner September 9, 2026 08:58

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable issues found.

Custom instructions used

  • .github/copilot-instructions.md
  • .github/instructions/reviewing.instructions.md

@achamayou
Amaury Chamayou (achamayou) merged commit 028fd5d into microsoft:main Sep 9, 2026
13 checks passed
@cjen1-msft
cjen1-msft deleted the snapshot-infra-01-disk-helpers branch September 10, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants