build: report cuda-bindings provenance in PEP 517 builds - #2520
Merged
rwgk merged 1 commit intoAug 6, 2026
Conversation
Resolve the CUDA path before importing cuda.bindings so the existing pathfinder import repairs PEP 517 namespace shadowing first. Reuse the resolved path for the CUDA include directory.
Contributor
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
rwgk
added
P1
This was referenced Aug 6, 2026
Contributor
Author
|
/ok to test |
|
juenglin
approved these changes
Aug 6, 2026
rwgk
deleted the
rwgk/stacked/ci/report_current_cuda_bindings_provenance
branch
August 6, 2026 16:09
rparolin
added a commit
to rparolin/cuda-python
that referenced
this pull request
Aug 12, 2026
Use pathlib and anchor build artifacts to build_hooks.py rather than the working directory, since a project can be built from anywhere. This covers the cythonize build_dir as well as the stamp: anchoring only one would let the two caches resolve to different places when building from outside cuda_core/. Force a rebuild when the stamp is missing too. A missing stamp means the last build's CUDA major is unknown, and on a first build forcing costs nothing because there are no artifacts to reuse. Move _check_build_major() after the cuda.bindings import. It re-enters _get_cuda_path() and reads cuda.h, which must not happen before the pathfinder import has repaired PEP 517 namespace shadowing (NVIDIA#2520); the major is not needed until cythonize(). Restores test_cuda_path_is_resolved_before_importing_bindings. Also restores full-test's 90 minute timeout, unrelated to this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rparolin
added a commit
that referenced
this pull request
Aug 27, 2026
* build: make Cython/extension artifacts CUDA-major aware Moving a checkout between the cu12 and cu13 pixi environments failed at compile time, pointing at code that is perfectly fine. Two independent caches are to blame, and neither tool notices the configuration changed: - Cython's up-to-date check hashes the .pyx and its cimport dependencies, but not compile_time_env. cuda.core feeds CUDA_CORE_BUILD_MAJOR through compile_time_env, so a cu13 -> cu12 switch silently reuses the cu13 generated C++. Reproduced on main: the cu12 build dies on 'CUdevWorkqueueConfigScope was not declared' in a build/cython/*.cpp generated under CUDA 13. - setuptools' build_ext compares source mtimes against the output .so. In an editable install that .so lives in the source tree under a name keyed by the Python ABI tag alone -- there is nowhere to record the CUDA major. So on a cu12 -> cu13 -> cu12 round trip the final build finds an older generated source next to a newer .so and skips the rebuild entirely. Both build backends now compute a build identity (CUDA major, plus the debug and coverage flags, which likewise change the generated C++ that neither tool tracks). Generated sources go to build/cython/<identity>, and build/.build-identity records the last completed build so setup.py can force build_ext when the configuration changes. Python version and platform stay out of the identity: setuptools already encodes them in its own build/lib.* and build/temp.* names. CUDA_PYTHON_COVERAGE keeps generating in-tree (build_dir=".") so it can still package the generated sources; it only contributes to the identity. Migration is self-healing. An existing unkeyed build/cython is orphaned and ignored; the first build after this change regenerates into a keyed directory and overwrites the in-tree extensions. Verified end to end with pixi on linux-64: cu13 -> cu12 -> cu13 for cuda_core. Before, the cu12 leg failed to compile; after, all three legs succeed, each major keeps its own build/cython/cu1X-debug directory, and the second cu13 extension is byte-identical to the first (md5 ae4090a4f66ab9cee67b5b2f64b43781), proving it was recompiled rather than left as the cu12 artifact. The new CI job covers cuda_core only. cuda_bindings cannot be source-built in its cu12 environment at all -- the 13.x sources reference CUDA 13-only symbols (CUatomicOperation, nvrtcBundledHeadersInfo, CUstreamCigCaptureParams), so even freshly generated cu12 sources fail against CUDA 12 headers. That is a pre-existing problem, unrelated to artifact staleness; the identical identity logic in cuda_bindings is covered by unit tests instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * build_hooks: match cuda_core's missing-cuda.h error message The two _determine_cuda_major_version implementations are annotated "keep in sync"; a missing cuda.h surfaced as a bare FileNotFoundError in cuda_bindings instead of the RuntimeError naming CUDA_PATH/CUDA_HOME. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * build: boil the CUDA-major fix down to essentials Cuts the change to the two mechanisms that are actually load-bearing for the cu12/cu13 problem, and drops everything that was speculative. Removed the debug and coverage identity axes. The stated justification for debug -- that gdb_debug changes the cythonize output -- is wrong: with gdb_debug=True the generated .c is byte-identical, and Cython only writes a side cython_debug/ directory. Keying by it produced a duplicate directory of identical sources and re-ran cythonize on every editable/wheel switch for no benefit. Coverage does change the generated C (linetrace), but toggling it is a separate defect from the one this PR is about. Removed the cuda_bindings half entirely. It has no compile_time_env, so its generated C is CUDA-major independent, and it cannot be source-built against CUDA 12 at all today -- the scenario the code guarded against is unreachable. That also drops a second _determine_cuda_major_version, a duplicated 50-line block, and a test file. What remains: cuda_core generates into build/cython/cu<major>, and build/.build-cuda-major records the last completed build so setup.py forces build_ext when the major changes. The stamp is a bare major rather than a composite identity string, so the "no build ran" guard and its test are gone too. Re-verified cu13 -> cu12 -> cu13 on linux-64: all three legs build, both build/cython/cu12 and build/cython/cu13 exist, and the second cu13 extension is byte-identical to the first (md5 1c53ab83a7201d54576175c912d03e93). 18 unit tests pass; pre-commit clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * tests: drop agent_authored markers from the new build-hook tests Author's call. The convention stays in CLAUDE.md and on the ~30 tests that already carry it; only the four added by this PR are affected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * build: address review feedback on CUDA-major-aware artifacts Use pathlib and anchor build artifacts to build_hooks.py rather than the working directory, since a project can be built from anywhere. This covers the cythonize build_dir as well as the stamp: anchoring only one would let the two caches resolve to different places when building from outside cuda_core/. Force a rebuild when the stamp is missing too. A missing stamp means the last build's CUDA major is unknown, and on a first build forcing costs nothing because there are no artifacts to reuse. Move _check_build_major() after the cuda.bindings import. It re-enters _get_cuda_path() and reads cuda.h, which must not happen before the pathfinder import has repaired PEP 517 namespace shadowing (#2520); the major is not needed until cythonize(). Restores test_cuda_path_is_resolved_before_importing_bindings. Also restores full-test's 90 minute timeout, unrelated to this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: cover the CUDA-major build wiring without a full build The cu13 -> cu12 -> cu13 CI job is the only thing exercising this fix, and it takes minutes. These check the same wiring in ~0.1s by replacing cythonize(), so nothing is generated or compiled: - generated sources for cu12 and cu13 land in different directories - that directory is anchored to build_hooks.py, so it agrees with the stamp regardless of the working directory - build_ext.finalize_options() picks up force_build_ext, and leaves force alone when it is clear Each was confirmed to fail when the corresponding fix is reverted. setup() is now guarded by __name__ == "__main__" so the command classes can be imported. setuptools always runs setup.py as a script, so builds are unaffected; verified with a clean rebuild. Note the sys.path handling in the cythonize helper: _build_cuda_core() permanently prepends cuda_bindings/ to sys.path, which makes setup.py's bare "import build_hooks" resolve to cuda_bindings' copy. The tests contain that mutation and pin the intended module. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related to #1824 and #2468.
Follow-up to #2509. This PR is intended to merge before #2510. #2510 separately enforces exact local-wheel selection, while this PR only makes the build hook's existing provenance reporting work consistently.
#2509 added a
cuda.corebuild-hook message that reports the importedcuda-bindingsversion and package directory. The message appears for previous-CTK CUDA 12 builds, but not for the 24 current-CTK CUDA 13 builds (see #2468 (comment) for a full report).The difference comes from call ordering in the PEP 517 in-tree backend.
cuda_core/build_hooks.pytries to importcuda.bindingsbefore calling_get_cuda_path(). At that point,backend-path = ["."]can leave the repository'scuda_core/cudadirectory cached as the only path in thecudanamespace, so the installed build dependency is not visible. The import failure is then suppressed by the existing best-effort guard._get_cuda_path()invokes the namespace repair documented in #1824, but currently does so only after that import attempt.CUDA 12 wheels happen not to expose the problem because they still contain the historical
_cuda_bindings_redirector.pth, which importscudaduring interpreter startup. That redirector stopped shipping with CUDA 13 as part of #792, explaining why only the previous-CTK builds currently reach the provenance print.This PR moves the existing
_get_cuda_path()call ahead of thecuda.bindingsimport and reuses its result later in the build. The namespace is therefore repaired before the import, allowing the existing #2509 message to report the CUDA 13 dependency's version and location and allowing the existing Cython path setup to use that imported package.This does not change build requirements or dependency-resolution policy.
_get_cuda_path()was already mandatory later in the same function, so the functional change is limited to performing that lookup earlier.