From 95bd1372bf6bae3162e9a6dd4bac594caa4b7915 Mon Sep 17 00:00:00 2001 From: Kasper Westman Date: Tue, 1 Sep 2026 22:39:37 +0200 Subject: [PATCH 1/3] Fixes bugs related to paths and differences in how clang-2100.1.1.101 outputs floating point numbers --- .claude/LESSONS.md | 12 ++- .../2026-09-01-f15-libcxx-profile.md | 73 +++++++++++++++++++ CHANGELOG.md | 15 ++++ dtwc/core/llfio_include.hpp | 6 +- scripts/test_f22_cpp_deprecations.py | 11 ++- tests/CMakeLists.txt | 2 +- .../test_distance_matrix_csv_contract.cmake | 4 +- tests/unit/unit_test_deterministic_series.cpp | 19 ++++- 8 files changed, 132 insertions(+), 10 deletions(-) create mode 100644 .claude/baselines/2026-09-01-f15-libcxx-profile.md diff --git a/.claude/LESSONS.md b/.claude/LESSONS.md index ef8ceed..a8c4b95 100644 --- a/.claude/LESSONS.md +++ b/.claude/LESSONS.md @@ -435,10 +435,14 @@ Critical knowledge to avoid repeating mistakes. `/fp:precise`; Clang without those relaxations matched MSVC. Linux libstdc++ produced a third result because the standard does not prescribe `uniform_real_distribution`'s engine-to-real mapping; there even the scalar - `[-1,1]` bytes differ. A behavior-neutral extraction must compare raw - IEEE-754 bytes under each verified compiler-plus-standard-library profile, - preserve the distribution type and draw schedule, and never relabel legacy - STL-distribution fixtures as `portable-v1`. + `[-1,1]` bytes differ. Apple Clang + libc++ is a fourth coherent row: + scalar/row `[-1,1]` hashes match libstdc++, but the continuous `[-10,10]` + stream differs by a few ULPs while the 3×3 full/band-0 oracles still hash + to the libstdc++ matrices. Register that row as `libcxx`; do not skip F15 + or replace the STL distribution. A behavior-neutral extraction must compare + raw IEEE-754 bytes under each verified compiler-plus-standard-library + profile, preserve the distribution type and draw schedule, and never + relabel legacy STL-distribution fixtures as `portable-v1`. - **Catch2 decomposition rejects unparenthesized logical OR.** An expression such as `CHECK((a && b) || (c && d))` reaches Catch2's deleted/decomposition guard and fails to compile; force the complete predicate to `bool` with one diff --git a/.claude/baselines/2026-09-01-f15-libcxx-profile.md b/.claude/baselines/2026-09-01-f15-libcxx-profile.md new file mode 100644 index 0000000..121edb1 --- /dev/null +++ b/.claude/baselines/2026-09-01-f15-libcxx-profile.md @@ -0,0 +1,73 @@ +# F15 libcxx fingerprint row — 2026-09-01 + +## Scope + +Apple Clang + libc++ is a fourth coherent F15 profile. It is not a skip and +not a generator change. Scalar and per-row `[-1,1]` hashes already matched +the registered `libstdcxx` schedule; the continuous `[-10,10]` accelerator +stream differed by a few ULPs, so `profile_for()` returned nullptr. + +Registered before the focused rebuild. Dump subject: production +`accelerator_series_set(3, 4, 42)`, then +`symmetric_zero_diagonal_matrix` with `dtwFull_L` and +`dtwBanded(..., 0)`, IEEE SHA-256 of little-endian binary64 bytes +exactly as `tests/unit/unit_test_deterministic_series.cpp`. + +## Environment [confirmed] + +```text +Apple clang version 21.0.0 (clang-2100.1.1.101) +STL: libc++ (Apple Command Line Tools) +CMAKE_BUILD_TYPE=Release +flags include: -O3 -fno-math-errno -fno-trapping-math -freciprocal-math + -fassociative-math -fno-signed-zeros -fno-rounding-math -march=native +host: darwin 25.6.0 +``` + +## Dumped fingerprints [confirmed] + +```text +scalar_hash=194FB0E76C52FCD84F09960547EEDC6A43788FDCC89F739DF44E3C49AF7B16E0 +rows_hash=1F9E6847BA0FFC7943EBCA024827CD6B5A890B8911BFF4F6C59211F3C28892AB +accelerator_hash=1D063EF12CB8680807CEEEF9F8C2F35331D0AA91186F824A16D9F3B2954AEF77 +full_hash=7DE312EABCFFB71D857BF97B9CFCE9C08A6F855E7CE25B863342BE46CBC38B73 +band0_hash=E81034CE0654315D254D07FA518DDCE7472A542A4EECBD740935E9DFF891022E +full_ne_band0=1 +``` + +Accelerator binary64 masks: + +```text +4017B933480DF696,C01953416B53F15F,40166012566A8F3E,3FFEFDF719BF921C, +BFF15561688E12F2,C0200041BE89C29E,BFEA14A986DAD398,C00A9B4B96F5696D, +C01C92166F9FE92A,4008246451B14AA8,C021BE586FDF2D37,4011C288EB7D97C0 +``` + +Scalar and row hashes match `libstdcxx`. Full and band-0 hashes match +`libstdcxx`. Band-0 differs from full, so the fixture remains non-degenerate. +The accelerator stream differs from `libstdcxx` at four ULP positions +(`…921C` vs `…9220`, `…12F2` vs `…12F0`, `…D398` vs `…D3A0`, `…696D` vs +`…696C`). Mixing those generator bits with another row's oracle hashes is a +fail. + +Fingerprint-row name: `libcxx` (not a compiler identity). + +## Pass band + +`unit_test_deterministic_series` must print + +```text +F15_TEST_SUPPORT profile=libcxx scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0 +``` + +and Catch2 `All tests passed` with the existing assertion/case floor. CTest +regex accepts `relaxed|precise|libstdcxx|libcxx`. + +## Focused execution [confirmed] + +```text +F15_TEST_SUPPORT profile=libcxx scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0 +All tests passed (165 assertions in 6 test cases) +``` + +`ctest -R '^unit_test_deterministic_series$'` → Passed. diff --git a/CHANGELOG.md b/CHANGELOG.md index 048eb96..b303ff7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,21 @@ This changelog contains a non-exhaustive list of new features and notable bug-fi

# Unreleased +- Registered the F15 `libcxx` deterministic-series fingerprint row for Apple + Clang + libc++. Scalar/row hashes match the existing libstdc++ schedule; + the continuous accelerator stream differs by a few ULPs and is now an + accepted coherent profile rather than a `profile == nullptr` failure. +- Fixed the F22 C++ deprecation probe so Apple Clang's paired + `-Xpreprocessor -fopenmp` (and `-Xclang -fopenmp`) is copied from + `compile_commands.json` instead of a bare `-fopenmp`, which Apple Clang + rejects as an unsupported option. Unix absolute paths such as `/Users/...` + are no longer treated as MSVC `/U` flags. LLFIO's libc++ + `char_traits` deprecation is ignored only inside + `llfio_include.hpp`, so `-Werror=deprecated-declarations` still diagnoses + DTWC++ aliases. +- Fixed the F14 distance-matrix CSV contract so path markers use a single + separator (`matrix_path` / `cache_path`) instead of a doubled slash, which + made the real CLI's POSIX output miss the registered exact-path check. - Enforced the frozen C++ deprecation policy for all 33 retained 1.x compatibility entities. `Problem::maxIter`/`N_repetition` and the seven legacy Problem I/O overloads now emit their registered replacement diff --git a/dtwc/core/llfio_include.hpp b/dtwc/core/llfio_include.hpp index f9a3bf1..0b0e199 100644 --- a/dtwc/core/llfio_include.hpp +++ b/dtwc/core/llfio_include.hpp @@ -7,9 +7,13 @@ // quickcpplib/ringbuffer_log.hpp installs an unbalanced Clang ignore for // -Wdeprecated-declarations on Windows. Preserve the caller's diagnostic state -// across the complete third-party include. +// across the complete third-party include. Apple libc++ also deprecates +// std::char_traits inside LLFIO headers; ignore that only while +// the dependency is parsed so a later -Werror=deprecated-declarations still +// diagnoses DTWC++ [[deprecated]] names (F22 / F45). #if defined(__clang__) # pragma clang diagnostic push +# pragma clang diagnostic ignored "-Wdeprecated-declarations" #endif #include diff --git a/scripts/test_f22_cpp_deprecations.py b/scripts/test_f22_cpp_deprecations.py index c397175..adcb7e7 100644 --- a/scripts/test_f22_cpp_deprecations.py +++ b/scripts/test_f22_cpp_deprecations.py @@ -379,6 +379,8 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]: if not arguments: raise RuntimeError("empty compile command") compiler = arguments[0].strip('"') + compiler_name = Path(compiler).name.lower() + msvc_slash_flags = compiler_name in ("cl", "cl.exe") or "clang-cl" in compiler_name context: list[str] = [] index = 1 paired_options = { @@ -388,7 +390,14 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]: "--sysroot", "-target", "--target", + # Apple Clang rejects a bare -fopenmp; CMake records + # -Xpreprocessor -fopenmp (and sometimes -Xclang -fopenmp). + "-Xpreprocessor", + "-Xclang", } + define_include_prefixes = ("-I", "-D", "-U") + if msvc_slash_flags: + define_include_prefixes += ("/I", "/D", "/U") while index < len(arguments): argument = arguments[index] if argument in paired_options: @@ -398,7 +407,7 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]: index += 2 continue if ( - argument.startswith(("-I", "/I", "-D", "/D", "-U", "/U")) + argument.startswith(define_include_prefixes) and '\\"' not in argument ): context.append(argument.strip('"')) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index c37bc24..f0ceed0 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -284,7 +284,7 @@ if(TARGET unit_test_deterministic_series) TEST unit_test_deterministic_series PROPERTY SKIP_RETURN_CODE) set(_f15_focused_regex - "F15_TEST_SUPPORT profile=(relaxed|precise|libstdcxx) scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0(.|[\r\n])*All tests passed \\((3[5-9]|[4-9][0-9]|[1-9][0-9][0-9]+) assertions in ([5-9]|[1-9][0-9]+) test cases\\)") + "F15_TEST_SUPPORT profile=(relaxed|precise|libstdcxx|libcxx) scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0(.|[\r\n])*All tests passed \\((3[5-9]|[4-9][0-9]|[1-9][0-9][0-9]+) assertions in ([5-9]|[1-9][0-9]+) test cases\\)") set_tests_properties( unit_test_deterministic_series PROPERTIES diff --git a/tests/integration/test_distance_matrix_csv_contract.cmake b/tests/integration/test_distance_matrix_csv_contract.cmake index cfb70d3..c6741a3 100644 --- a/tests/integration/test_distance_matrix_csv_contract.cmake +++ b/tests/integration/test_distance_matrix_csv_contract.cmake @@ -166,7 +166,7 @@ function(run_cli_route route output_dir threshold expect_mmap) "${route} distance-matrix output") require_occurrences( "${normalized_stdout}" - "\"${output_dir}//conformance_distance_matrix.csv\"" 1 + "\"${matrix_path}\"" 1 "${route} exact distance-matrix path") foreach(execution_marker IN ITEMS "DTWC++ Clustering" @@ -185,7 +185,7 @@ function(run_cli_route route output_dir threshold expect_mmap) "${route} mmap route") require_occurrences( "${normalized_stdout}" - "\"${output_dir}//conformance_distmat.cache\"" 1 + "\"${cache_path}\"" 1 "${route} exact mmap cache") if(NOT EXISTS "${cache_path}") message(FATAL_ERROR diff --git a/tests/unit/unit_test_deterministic_series.cpp b/tests/unit/unit_test_deterministic_series.cpp index b08b709..651b901 100644 --- a/tests/unit/unit_test_deterministic_series.cpp +++ b/tests/unit/unit_test_deterministic_series.cpp @@ -108,6 +108,21 @@ constexpr Profile kLibstdcxx{ } }; +constexpr Profile kLibcxx{ + "libcxx", + "1D063EF12CB8680807CEEEF9F8C2F35331D0AA91186F824A16D9F3B2954AEF77", + "7DE312EABCFFB71D857BF97B9CFCE9C08A6F855E7CE25B863342BE46CBC38B73", + "E81034CE0654315D254D07FA518DDCE7472A542A4EECBD740935E9DFF891022E", + { + UINT64_C(0x4017B933480DF696), UINT64_C(0xC01953416B53F15F), + UINT64_C(0x40166012566A8F3E), UINT64_C(0x3FFEFDF719BF921C), + UINT64_C(0xBFF15561688E12F2), UINT64_C(0xC0200041BE89C29E), + UINT64_C(0xBFEA14A986DAD398), UINT64_C(0xC00A9B4B96F5696D), + UINT64_C(0xC01C92166F9FE92A), UINT64_C(0x4008246451B14AA8), + UINT64_C(0xC021BE586FDF2D37), UINT64_C(0x4011C288EB7D97C0), + } +}; + void update_little_endian_double( dtwc::core::detail::Sha256 &sha, double value) { @@ -154,6 +169,7 @@ const Profile *profile_for(std::string_view accelerator_hash) if (accelerator_hash == kRelaxed.accelerator_hash) return &kRelaxed; if (accelerator_hash == kPrecise.accelerator_hash) return &kPrecise; if (accelerator_hash == kLibstdcxx.accelerator_hash) return &kLibstdcxx; + if (accelerator_hash == kLibcxx.accelerator_hash) return &kLibcxx; return nullptr; } @@ -162,7 +178,8 @@ const Profile *coherent_profile( std::string_view full_hash, std::string_view band0_hash) { - for (const Profile *profile : {&kRelaxed, &kPrecise, &kLibstdcxx}) { + for (const Profile *profile : + {&kRelaxed, &kPrecise, &kLibstdcxx, &kLibcxx}) { if (accelerator_hash == profile->accelerator_hash && full_hash == profile->full_hash && band0_hash == profile->band0_hash) { From 73084e161b8aa59282a469b2f1dffd39ad72674f Mon Sep 17 00:00:00 2001 From: Kasper Westman Date: Tue, 1 Sep 2026 23:59:36 +0200 Subject: [PATCH 2/3] Fixes oracle/Fp issues also for HPC compiler --- .claude/LESSONS.md | 13 ++++++ CHANGELOG.md | 9 +++++ PLAN.md | 9 +++++ dtwc/Problem.hpp | 21 ++++++++++ .../test_fast_clara_parquet_parity.cmake | 40 ++++++++++++++++--- .../unit_test_nearest_medoid_assignment.cpp | 20 ++++++++-- 6 files changed, 103 insertions(+), 9 deletions(-) diff --git a/.claude/LESSONS.md b/.claude/LESSONS.md index a8c4b95..560c443 100644 --- a/.claude/LESSONS.md +++ b/.claude/LESSONS.md @@ -443,6 +443,19 @@ Critical knowledge to avoid repeating mistakes. raw IEEE-754 bytes under each verified compiler-plus-standard-library profile, preserve the distribution type and draw schedule, and never relabel legacy STL-distribution fixtures as `portable-v1`. +- **A last-bit pin is not portable across GCC vs Apple Clang under the + Release reassociation set.** Arrhenius `LastTest.log` (2026-09-01) failed + three gates that were green on Apple Clang: F13 pinned `nearest[]` bits + `0x8000000000000000` (`-0.0`) while GCC flushed them to `+0.0`; F22's + canonical probe treated GCC's constructor-NSDMI use of deprecated + `maxIter`/`N_repetition` as a canonical diagnostic; F8 Soft-DTW + `total_cost` differed by 2 ULP (`CC…` vs `CE…`) after resident≡stream and + labels/medoids already matched. Canonicalize signed zeros in the F13 + oracle the same way as the zero objective; pragma-silence *constructors* + (not the fields) in `Problem.hpp`; register a second Soft-DTW cost encoding + and keep resident≡stream byte-identity. Do not skip the tests, do not + loosen Standard DTW goldens, and do not remove `[[deprecated]]` from the + fields. Evidence: `.claude/baselines/2026-09-01-hpc-gcc-fp-and-deprecation.md`. - **Catch2 decomposition rejects unparenthesized logical OR.** An expression such as `CHECK((a && b) || (c && d))` reaches Catch2's deleted/decomposition guard and fails to compile; force the complete predicate to `bool` with one diff --git a/CHANGELOG.md b/CHANGELOG.md index b303ff7..3e5e787 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,15 @@ This changelog contains a non-exhaustive list of new features and notable bug-fi

# Unreleased +- Canonical construction of `Problem` no longer diagnoses the deprecated + `maxIter` / `N_repetition` fields under GCC's constructor NSDMI check, so + the F22 canonical probe stays silent while caller access of those fields + still warns. The F13 assignment oracle now stores exact-zero nearest/second + distances as `+0.0`, matching the existing zero-objective rule, because + `-fno-signed-zeros` lets GCC flush `-0.0`. F8 Soft-DTW resident/stream + parity still requires byte-identical checkpoints and exact labels/medoids, + and now accepts a registered 2-ULP GCC encoding of `total_cost` alongside + the original MSVC/Apple Clang bytes. - Registered the F15 `libcxx` deterministic-series fingerprint row for Apple Clang + libc++. Scalar/row hashes match the existing libstdc++ schedule; the continuous accelerator stream differs by a few ULPs and is now an diff --git a/PLAN.md b/PLAN.md index 2b259f2..5027d7b 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1164,3 +1164,12 @@ colour system transfer verbatim**. clustering (a)–(f). AGENTS.md floors updated to the current 122/122/122/124 CTest and 1009/12/1-expected-F39-red Python inventories. Cursor unchanged: F22 final serial full-gate adjudication, then F23 per the cadence. +- 2026-09-01 (HPC GCC vs Apple Clang gate repairs): Arrhenius + `LastTest.log` failed F13 signed-zero nearest bits, F22 canonical + `Problem` construction, and F8 Soft-DTW `total_cost` last bits (2 ULP) + after F14/F15 from `95bd137` already passed. Binding repairs: F13 oracle + stores exact-zero nearest/second as `+0.0`; F22 silences GCC constructor + NSDMI diagnostics around `Problem` constructors only; F8 accepts + `CE31540B20B024C0` as a second Soft-DTW cost encoding while resident≡stream + and labels/medoids stay exact. Do not skip these tests. Evidence: + `.claude/baselines/2026-09-01-hpc-gcc-fp-and-deprecation.md`. diff --git a/dtwc/Problem.hpp b/dtwc/Problem.hpp index 53b9a26..244d085 100644 --- a/dtwc/Problem.hpp +++ b/dtwc/Problem.hpp @@ -276,6 +276,20 @@ class Problem std::vector centroids_ind; //!< indices of cluster centroids. [0, Np) // Constructors: + // GCC emits -Wdeprecated-declarations for in-class initializers of the + // deprecated maxIter / N_repetition fields at every constructor definition. + // Canonical construction must stay silent (F22); caller access of those + // fields must still diagnose. Same push/pop as Problem.cpp accessors. +#if defined(__clang__) +# pragma clang diagnostic push +# pragma clang diagnostic ignored "-Wdeprecated-declarations" +#elif defined(__GNUC__) +# pragma GCC diagnostic push +# pragma GCC diagnostic ignored "-Wdeprecated-declarations" +#elif defined(_MSC_VER) +# pragma warning(push) +# pragma warning(disable : 4996) +#endif Problem() { rebind_dtw_fn(); } Problem(std::string_view problem_name) : name_{ problem_name } { @@ -287,6 +301,13 @@ class Problem adopt_loaded_data(loader.load_stored()); refresh_distance_matrix(); // also calls rebind_dtw_fn() } +#if defined(__clang__) +# pragma clang diagnostic pop +#elif defined(__GNUC__) +# pragma GCC diagnostic pop +#elif defined(_MSC_VER) +# pragma warning(pop) +#endif Problem(const Problem &) = delete; Problem &operator=(const Problem &) = delete; Problem(Problem &&); diff --git a/tests/integration/test_fast_clara_parquet_parity.cmake b/tests/integration/test_fast_clara_parquet_parity.cmake index 456b6fe..f2160c4 100644 --- a/tests/integration/test_fast_clara_parquet_parity.cmake +++ b/tests/integration/test_fast_clara_parquet_parity.cmake @@ -211,13 +211,19 @@ function(check_f8_config file(SHA256 "${resident_dir}/parity_checkpoint.bin" checkpoint_sha) string(TOUPPER "${checkpoint_sha}" checkpoint_sha) + # expected_cost_hex_le / _be may be a semicolon-separated list of accepted + # IEEE-754 encodings. Resident and stream must still match each other + # (compare_files above) and both land in that set. Standard DTW stays a + # single encoding; Soft-DTW registers a 2-ULP GCC profile (see the + # f64_softdtw call site). if(BYTE_ORDER STREQUAL "LITTLE_ENDIAN") - set(expected_cost_hex "${expected_cost_hex_le}") + set(accepted_cost_hexes "${expected_cost_hex_le}") elseif(BYTE_ORDER STREQUAL "BIG_ENDIAN") - set(expected_cost_hex "${expected_cost_hex_be}") + set(accepted_cost_hexes "${expected_cost_hex_be}") else() message(FATAL_ERROR "F8 unknown C++ byte order '${BYTE_ORDER}'") endif() + set(observed_cost_hex "") foreach(mode IN ITEMS resident stream) file(READ "${${mode}_dir}/parity_checkpoint.bin" @@ -226,12 +232,23 @@ function(check_f8_config LIMIT 8 HEX) string(TOUPPER "${cost_hex}" cost_hex) - if(NOT cost_hex STREQUAL expected_cost_hex) + if(observed_cost_hex STREQUAL "") + set(observed_cost_hex "${cost_hex}") + elseif(NOT cost_hex STREQUAL observed_cost_hex) + message(FATAL_ERROR + "F8 ${config_id} resident/stream total_cost split: " + "${observed_cost_hex} vs ${cost_hex}") + endif() + list(FIND accepted_cost_hexes "${cost_hex}" accepted_index) + if(accepted_index EQUAL -1) message(FATAL_ERROR "F8 ${config_id}/${mode} total_cost bytes=${cost_hex}, " - "expected=${expected_cost_hex}") + "accepted=${accepted_cost_hexes}") endif() endforeach() + if(config_id STREQUAL "f64_softdtw") + set(f64_softdtw_cost_hex "${observed_cost_hex}" PARENT_SCOPE) + endif() if(WIN32) file(SHA256 "${resident_dir}/parity_labels.csv" labels_sha) @@ -264,9 +281,16 @@ check_f8_config( f32_standard float32 standard none 4.4 000000EC99991140 40119999EC000000 EEA65070341F900BA212909242AD54FE7A48F1E4A94704864CBF77C89FC28FC1) +# Soft-DTW total_cost encodings registered before the HPC re-run: +# CC31540B20B024C0 / C024B0200B5431CC = -10.343994478252078 +# (MSVC / Apple Clang baseline, F8 2026-07-23) +# CE31540B20B024C0 / C024B0200B5431CE = -10.343994478252082 +# (GCC 14 + -fassociative-math, 2 ULP; Arrhenius LastTest.log 2026-09-01) +# Labels/medoids stay exact. Resident≡stream stays byte-identical. check_f8_config( f64_softdtw float64 softdtw 0.7 -10.344 - CC31540B20B024C0 C024B0200B5431CC + "CC31540B20B024C0;CE31540B20B024C0" + "C024B0200B5431CC;C024B0200B5431CE" 67D818D58370CB6E17E4E8922175A343ADE2D53C7873A62522FE9D3C14734B79) if(f64_standard_checkpoint_sha STREQUAL f32_standard_checkpoint_sha @@ -295,7 +319,11 @@ if(NOT run_count EQUAL 6 "pairs=${pair_count} configs=${config_count}") endif() +if(NOT DEFINED f64_softdtw_cost_hex OR f64_softdtw_cost_hex STREQUAL "") + message(FATAL_ERROR "F8 missing observed Soft-DTW total_cost hex") +endif() message(STATUS "F8_PARITY subject=real_dtwc_cl runs=${run_count} " "route_markers=${route_check_count}/12 parity=${pair_count}/9 " - "configs=${config_count}/3_distinct fixture_sha256=${fixture_sha}") + "configs=${config_count}/3_distinct fixture_sha256=${fixture_sha} " + "softdtw_cost=${f64_softdtw_cost_hex}") diff --git a/tests/unit/algorithms/unit_test_nearest_medoid_assignment.cpp b/tests/unit/algorithms/unit_test_nearest_medoid_assignment.cpp index 74fc721..2acc564 100644 --- a/tests/unit/algorithms/unit_test_nearest_medoid_assignment.cpp +++ b/tests/unit/algorithms/unit_test_nearest_medoid_assignment.cpp @@ -92,8 +92,21 @@ OracleAssignment independent_assignment_oracle( }); result.labels[point] = static_cast(candidates[0].slot); - result.nearest[point] = candidates[0].distance; - if (k > 1) result.second[point] = candidates[1].distance; + // The Release flag set includes -fno-signed-zeros. GCC may flush -0.0 to + // +0.0 on a copy; Apple Clang often preserves the sign bit. The F13 + // contract already canonicalizes a zero *objective* to +0.0. Do the same + // for stored nearest/second distances so the oracle is not a signed-zero + // compiler fingerprint. + volatile double nearest = candidates[0].distance; + if (nearest == 0.0) + nearest = 0.0; + result.nearest[point] = nearest; + if (k > 1) { + volatile double second = candidates[1].distance; + if (second == 0.0) + second = 0.0; + result.second[point] = second; + } const double next = total + candidates[0].distance; if (!std::isfinite(next)) { @@ -239,7 +252,7 @@ TEST_CASE("F13 independent oracle pins ties, presence, and ordered bits", UINT64_C(0x4340000000000000), UINT64_C(0x0000000000000000), UINT64_C(0x3ff0000000000000), - UINT64_C(0x8000000000000000), + UINT64_C(0x0000000000000000), UINT64_C(0xc340000000000000), UINT64_C(0x4000000000000000), }; @@ -249,6 +262,7 @@ TEST_CASE("F13 independent oracle pins ties, presence, and ordered bits", const auto zero = independent_assignment_oracle({{-0.0, +0.0}}); REQUIRE(zero.labels == std::vector{0}); + REQUIRE(bits(zero.nearest[0]) == UINT64_C(0x0000000000000000)); REQUIRE(bits(zero.objective) == UINT64_C(0x0000000000000000)); const double maximum = std::numeric_limits::max(); From f13ddc037665d1e42ae4da448b4c6ed5d7fe1fcb Mon Sep 17 00:00:00 2001 From: Kasper Westman Date: Wed, 2 Sep 2026 10:51:04 +0200 Subject: [PATCH 3/3] Make sure highs gets added as a static library when building for python --- .claude/LESSONS.md | 6 +++++ CHANGELOG.md | 3 +++ cmake/Dependencies.cmake | 58 +++++++++++++++++++++++++++++----------- 3 files changed, 51 insertions(+), 16 deletions(-) diff --git a/.claude/LESSONS.md b/.claude/LESSONS.md index 560c443..1eaea49 100644 --- a/.claude/LESSONS.md +++ b/.claude/LESSONS.md @@ -37,6 +37,12 @@ Critical knowledge to avoid repeating mistakes. ## Python packaging / bindings +- **A wheel repair tool does not protect local source/editable installs.** + HiGHS defaults to a shared library on Unix, leaving the installed Python + extension with `@rpath/libhighs.1.dylib` but no `LC_RPATH`; cibuildwheel can + repair release wheels, while `uv pip install -e .` cannot. Build HiGHS and + its extras as scoped PIC static libraries for Python packaging, and assert + the resulting CMake target type during configuration. - **Never infer Python import provenance from the checkout layout.** The 2026-07-23 probe found a mixed environment: `dtwcpp` and `_api.py` resolve to `python/dtwcpp/` in the repository, while `_dtwcpp_core` resolves to the venv diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e5e787..6b8d1c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,9 @@ This changelog contains a non-exhaustive list of new features and notable bug-fi

# Unreleased +- Fixed macOS Python source and editable installs by statically bundling HiGHS + into the native extension, eliminating the unresolved + `@rpath/libhighs.1.dylib` import dependency. - Canonical construction of `Problem` no longer diagnoses the deprecated `maxIter` / `N_repetition` fields under GCC's constructor NSDMI check, so the F22 canonical probe stays silent while caller access of those fields diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake index d27b3e5..7c095a4 100644 --- a/cmake/Dependencies.cmake +++ b/cmake/Dependencies.cmake @@ -39,22 +39,48 @@ function(dtwc_setup_dependencies) else() set(CUPDLP_GPU OFF CACHE BOOL "Enable HiGHS cuPDLP GPU support" FORCE) endif() - CPMAddPackage( - NAME highs - URL "https://github.com/ERGO-Code/HiGHS/archive/refs/tags/v1.15.1.tar.gz" - # SHA256 pinned (Task 0.12). Computed 2026-07-08 from the GitHub release - # tarball for the immutable tag v1.15.1 (`curl -sL … | sha256sum`). - URL_HASH SHA256=a840d269dff2fafb371dd247df13ad5e026d7ce3b35ad3dc1eedd59bf0c2fb16 - SYSTEM - EXCLUDE_FROM_ALL - OPTIONS - "CI OFF" - "ZLIB OFF" - "BUILD_CXX_EXE OFF" - "BUILD_EXAMPLES OFF" - "BUILD_TESTING OFF" - "FAST_BUILD ON" - ) + # A Python extension cannot rely on HiGHS' Unix default of BUILD_SHARED_LIBS=ON: + # the resulting @rpath/libhighs dependency is outside site-packages in local + # source/editable installs. Keep these normal variables scoped to the HiGHS + # subproject so native DTWC++ builds retain their requested shared-library + # policy. HiGHS sets PIC on its static target, making it safe to fold into the + # extension. GPU HiGHS is exempt because its Windows CUDA build requires DLLs. + block(SCOPE_FOR VARIABLES) + if(DTWC_BUILD_PYTHON AND NOT DTWC_HIGHS_GPU) + set(BUILD_SHARED_LIBS OFF) + set(BUILD_SHARED_EXTRAS_LIB OFF) + endif() + + CPMAddPackage( + NAME highs + URL "https://github.com/ERGO-Code/HiGHS/archive/refs/tags/v1.15.1.tar.gz" + # SHA256 pinned (Task 0.12). Computed 2026-07-08 from the GitHub release + # tarball for the immutable tag v1.15.1 (`curl -sL … | sha256sum`). + URL_HASH SHA256=a840d269dff2fafb371dd247df13ad5e026d7ce3b35ad3dc1eedd59bf0c2fb16 + SYSTEM + EXCLUDE_FROM_ALL + OPTIONS + "CI OFF" + "ZLIB OFF" + "BUILD_CXX_EXE OFF" + "BUILD_EXAMPLES OFF" + "BUILD_TESTING OFF" + "FAST_BUILD ON" + ) + endblock() + + # Fail during configuration, rather than shipping another extension with an + # unresolved libhighs dependency, if HiGHS changes how it honours + # BUILD_SHARED_LIBS. + if(DTWC_BUILD_PYTHON AND NOT DTWC_HIGHS_GPU AND TARGET highs) + get_target_property(_dtwc_highs_library_type highs TYPE) + if(NOT _dtwc_highs_library_type STREQUAL "STATIC_LIBRARY") + message(FATAL_ERROR + "Python packages require statically bundled HiGHS; got " + "${_dtwc_highs_library_type}") + endif() + endif() + # Historically HiGHS <=1.14.0 had a debug assertion (ub_consistent) that # fired on valid warm-start MIP solves (primal-dual integral bookkeeping # tolerance 1e-12 too tight after a presolve reset). Retained defensively: