Skip to content

[CI] Validate pull requests against the Genesis unit tests and benchmarks. - #909

Open
duburcqa wants to merge 1 commit into
mainfrom
duburcqa/genesis_cross_validation
Open

duburcqa wants to merge 1 commit into
mainfrom
duburcqa/genesis_cross_validation

Conversation

@duburcqa

@duburcqa duburcqa commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Issue: #

Brief Summary

Genesis is the main consumer of Quadrants, and its test suite and benchmarks exercise the compiler far beyond what the tests of this repository cover. This adds a Genesis workflow that validates every pull request against the default branch of Genesis, the way the Genesis repository validates its own pull requests against the Nyx plugin.

The workflow starts once "Manylinux wheel build" completes with success and calls the CI of Genesis itself, made reusable by Genesis-Embodied-AI/genesis-world#3371, on Genesis main installed with the wheel built from the revision under test:

  • the minimal set of Genesis unit tests (required and not slow, dynamic array mode, GPU backend);
  • every Genesis benchmark, in both dynamic array and field modes;
  • the comparison of the benchmarks against the history of Genesis main, with the same script and tolerances as the Genesis CI, which comments on the pull request in case of regression or abnormal deviation. Nothing is uploaded to W&B.

The outcome is published as two checks on the commit under test: "Genesis Unit Tests & Benchmarks" and "Genesis Benchmark Comparison" (comparison tables).

Walkthrough

  • The orchestration lives in production.yml and alarm.yml of the Genesis repository, called as reusable workflows. This file only carries the trigger, the run id of the wheel build, the test selection, and the publication of the check.
  • The Genesis side vets the call (a pull request against the default branch of this repository, forks included), picks the wheel fitting its environment among the artifacts of the run, and routes the call to the one Genesis runner labelled for calls, so the other stays with the Genesis CI and calls queue in order. A newer revision of a pull request cancels its stale run here.

Prerequisites outside this repository:

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5936cd2bc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .github/workflows/genesis.yml Outdated
Comment thread .github/workflows/genesis.yml Outdated
Comment thread .github/workflows/genesis.yml Outdated
@duburcqa
duburcqa marked this pull request as draft September 14, 2026 17:14
@duburcqa
duburcqa force-pushed the duburcqa/genesis_cross_validation branch from d5936cd to 2f3291d Compare September 14, 2026 17:19
@duburcqa
duburcqa force-pushed the duburcqa/genesis_cross_validation branch from 2f3291d to 183f323 Compare September 14, 2026 17:33
@github-actions

Copy link
Copy Markdown

@duburcqa
duburcqa force-pushed the duburcqa/genesis_cross_validation branch from 183f323 to 67ba4ed Compare September 15, 2026 13:20
@duburcqa
duburcqa marked this pull request as ready for review September 15, 2026 14:03
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant