Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions .claude/LESSONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ Critical knowledge to avoid repeating mistakes.

## Python packaging / bindings

- **A wheel repair tool does not protect local source/editable installs.**
HiGHS defaults to a shared library on Unix, leaving the installed Python
extension with `@rpath/libhighs.1.dylib` but no `LC_RPATH`; cibuildwheel can
repair release wheels, while `uv pip install -e .` cannot. Build HiGHS and
its extras as scoped PIC static libraries for Python packaging, and assert
the resulting CMake target type during configuration.
- **Never infer Python import provenance from the checkout layout.** The
2026-07-23 probe found a mixed environment: `dtwcpp` and `_api.py` resolve to
`python/dtwcpp/` in the repository, while `_dtwcpp_core` resolves to the venv
Expand Down Expand Up @@ -435,10 +441,27 @@ Critical knowledge to avoid repeating mistakes.
`/fp:precise`; Clang without those relaxations matched MSVC. Linux
libstdc++ produced a third result because the standard does not prescribe
`uniform_real_distribution`'s engine-to-real mapping; there even the scalar
`[-1,1]` bytes differ. A behavior-neutral extraction must compare raw
IEEE-754 bytes under each verified compiler-plus-standard-library profile,
preserve the distribution type and draw schedule, and never relabel legacy
STL-distribution fixtures as `portable-v1`.
`[-1,1]` bytes differ. Apple Clang + libc++ is a fourth coherent row:
scalar/row `[-1,1]` hashes match libstdc++, but the continuous `[-10,10]`
stream differs by a few ULPs while the 3×3 full/band-0 oracles still hash
to the libstdc++ matrices. Register that row as `libcxx`; do not skip F15
or replace the STL distribution. A behavior-neutral extraction must compare
raw IEEE-754 bytes under each verified compiler-plus-standard-library
profile, preserve the distribution type and draw schedule, and never
relabel legacy STL-distribution fixtures as `portable-v1`.
- **A last-bit pin is not portable across GCC vs Apple Clang under the
Release reassociation set.** Arrhenius `LastTest.log` (2026-09-01) failed
three gates that were green on Apple Clang: F13 pinned `nearest[]` bits
`0x8000000000000000` (`-0.0`) while GCC flushed them to `+0.0`; F22's
canonical probe treated GCC's constructor-NSDMI use of deprecated
`maxIter`/`N_repetition` as a canonical diagnostic; F8 Soft-DTW
`total_cost` differed by 2 ULP (`CC…` vs `CE…`) after resident≡stream and
labels/medoids already matched. Canonicalize signed zeros in the F13
oracle the same way as the zero objective; pragma-silence *constructors*
(not the fields) in `Problem.hpp`; register a second Soft-DTW cost encoding
and keep resident≡stream byte-identity. Do not skip the tests, do not
loosen Standard DTW goldens, and do not remove `[[deprecated]]` from the
fields. Evidence: `.claude/baselines/2026-09-01-hpc-gcc-fp-and-deprecation.md`.
- **Catch2 decomposition rejects unparenthesized logical OR.** An expression
such as `CHECK((a && b) || (c && d))` reaches Catch2's deleted/decomposition
guard and fails to compile; force the complete predicate to `bool` with one
Expand Down
73 changes: 73 additions & 0 deletions .claude/baselines/2026-09-01-f15-libcxx-profile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# F15 libcxx fingerprint row — 2026-09-01

## Scope

Apple Clang + libc++ is a fourth coherent F15 profile. It is not a skip and
not a generator change. Scalar and per-row `[-1,1]` hashes already matched
the registered `libstdcxx` schedule; the continuous `[-10,10]` accelerator
stream differed by a few ULPs, so `profile_for()` returned nullptr.

Registered before the focused rebuild. Dump subject: production
`accelerator_series_set(3, 4, 42)`, then
`symmetric_zero_diagonal_matrix` with `dtwFull_L<double>` and
`dtwBanded<double>(..., 0)`, IEEE SHA-256 of little-endian binary64 bytes
exactly as `tests/unit/unit_test_deterministic_series.cpp`.

## Environment [confirmed]

```text
Apple clang version 21.0.0 (clang-2100.1.1.101)
STL: libc++ (Apple Command Line Tools)
CMAKE_BUILD_TYPE=Release
flags include: -O3 -fno-math-errno -fno-trapping-math -freciprocal-math
-fassociative-math -fno-signed-zeros -fno-rounding-math -march=native
host: darwin 25.6.0
```

## Dumped fingerprints [confirmed]

```text
scalar_hash=194FB0E76C52FCD84F09960547EEDC6A43788FDCC89F739DF44E3C49AF7B16E0
rows_hash=1F9E6847BA0FFC7943EBCA024827CD6B5A890B8911BFF4F6C59211F3C28892AB
accelerator_hash=1D063EF12CB8680807CEEEF9F8C2F35331D0AA91186F824A16D9F3B2954AEF77
full_hash=7DE312EABCFFB71D857BF97B9CFCE9C08A6F855E7CE25B863342BE46CBC38B73
band0_hash=E81034CE0654315D254D07FA518DDCE7472A542A4EECBD740935E9DFF891022E
full_ne_band0=1
```

Accelerator binary64 masks:

```text
4017B933480DF696,C01953416B53F15F,40166012566A8F3E,3FFEFDF719BF921C,
BFF15561688E12F2,C0200041BE89C29E,BFEA14A986DAD398,C00A9B4B96F5696D,
C01C92166F9FE92A,4008246451B14AA8,C021BE586FDF2D37,4011C288EB7D97C0
```

Scalar and row hashes match `libstdcxx`. Full and band-0 hashes match
`libstdcxx`. Band-0 differs from full, so the fixture remains non-degenerate.
The accelerator stream differs from `libstdcxx` at four ULP positions
(`…921C` vs `…9220`, `…12F2` vs `…12F0`, `…D398` vs `…D3A0`, `…696D` vs
`…696C`). Mixing those generator bits with another row's oracle hashes is a
fail.

Fingerprint-row name: `libcxx` (not a compiler identity).

## Pass band

`unit_test_deterministic_series` must print

```text
F15_TEST_SUPPORT profile=libcxx scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0
```

and Catch2 `All tests passed` with the existing assertion/case floor. CTest
regex accepts `relaxed|precise|libstdcxx|libcxx`.

## Focused execution [confirmed]

```text
F15_TEST_SUPPORT profile=libcxx scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0
All tests passed (165 assertions in 6 test cases)
```

`ctest -R '^unit_test_deterministic_series$'` → Passed.
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,33 @@ This changelog contains a non-exhaustive list of new features and notable bug-fi
<br/><br/>
# Unreleased

- Fixed macOS Python source and editable installs by statically bundling HiGHS
into the native extension, eliminating the unresolved
`@rpath/libhighs.1.dylib` import dependency.
- Canonical construction of `Problem` no longer diagnoses the deprecated
`maxIter` / `N_repetition` fields under GCC's constructor NSDMI check, so
the F22 canonical probe stays silent while caller access of those fields
still warns. The F13 assignment oracle now stores exact-zero nearest/second
distances as `+0.0`, matching the existing zero-objective rule, because
`-fno-signed-zeros` lets GCC flush `-0.0`. F8 Soft-DTW resident/stream
parity still requires byte-identical checkpoints and exact labels/medoids,
and now accepts a registered 2-ULP GCC encoding of `total_cost` alongside
the original MSVC/Apple Clang bytes.
- Registered the F15 `libcxx` deterministic-series fingerprint row for Apple
Clang + libc++. Scalar/row hashes match the existing libstdc++ schedule;
the continuous accelerator stream differs by a few ULPs and is now an
accepted coherent profile rather than a `profile == nullptr` failure.
- Fixed the F22 C++ deprecation probe so Apple Clang's paired
`-Xpreprocessor -fopenmp` (and `-Xclang -fopenmp`) is copied from
`compile_commands.json` instead of a bare `-fopenmp`, which Apple Clang
rejects as an unsupported option. Unix absolute paths such as `/Users/...`
are no longer treated as MSVC `/U` flags. LLFIO's libc++
`char_traits<std::byte>` deprecation is ignored only inside
`llfio_include.hpp`, so `-Werror=deprecated-declarations` still diagnoses
DTWC++ aliases.
- Fixed the F14 distance-matrix CSV contract so path markers use a single
separator (`matrix_path` / `cache_path`) instead of a doubled slash, which
made the real CLI's POSIX output miss the registered exact-path check.
- Enforced the frozen C++ deprecation policy for all 33 retained 1.x
compatibility entities. `Problem::maxIter`/`N_repetition` and the seven
legacy Problem I/O overloads now emit their registered replacement
Expand Down
9 changes: 9 additions & 0 deletions PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1164,3 +1164,12 @@ colour system transfer verbatim**.
clustering (a)–(f). AGENTS.md floors updated to the current 122/122/122/124
CTest and 1009/12/1-expected-F39-red Python inventories. Cursor unchanged:
F22 final serial full-gate adjudication, then F23 per the cadence.
- 2026-09-01 (HPC GCC vs Apple Clang gate repairs): Arrhenius
`LastTest.log` failed F13 signed-zero nearest bits, F22 canonical
`Problem` construction, and F8 Soft-DTW `total_cost` last bits (2 ULP)
after F14/F15 from `95bd137` already passed. Binding repairs: F13 oracle
stores exact-zero nearest/second as `+0.0`; F22 silences GCC constructor
NSDMI diagnostics around `Problem` constructors only; F8 accepts
`CE31540B20B024C0` as a second Soft-DTW cost encoding while resident≡stream
and labels/medoids stay exact. Do not skip these tests. Evidence:
`.claude/baselines/2026-09-01-hpc-gcc-fp-and-deprecation.md`.
58 changes: 42 additions & 16 deletions cmake/Dependencies.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -39,22 +39,48 @@ function(dtwc_setup_dependencies)
else()
set(CUPDLP_GPU OFF CACHE BOOL "Enable HiGHS cuPDLP GPU support" FORCE)
endif()
CPMAddPackage(
NAME highs
URL "https://github.com/ERGO-Code/HiGHS/archive/refs/tags/v1.15.1.tar.gz"
# SHA256 pinned (Task 0.12). Computed 2026-07-08 from the GitHub release
# tarball for the immutable tag v1.15.1 (`curl -sL … | sha256sum`).
URL_HASH SHA256=a840d269dff2fafb371dd247df13ad5e026d7ce3b35ad3dc1eedd59bf0c2fb16
SYSTEM
EXCLUDE_FROM_ALL
OPTIONS
"CI OFF"
"ZLIB OFF"
"BUILD_CXX_EXE OFF"
"BUILD_EXAMPLES OFF"
"BUILD_TESTING OFF"
"FAST_BUILD ON"
)
# A Python extension cannot rely on HiGHS' Unix default of BUILD_SHARED_LIBS=ON:
# the resulting @rpath/libhighs dependency is outside site-packages in local
# source/editable installs. Keep these normal variables scoped to the HiGHS
# subproject so native DTWC++ builds retain their requested shared-library
# policy. HiGHS sets PIC on its static target, making it safe to fold into the
# extension. GPU HiGHS is exempt because its Windows CUDA build requires DLLs.
block(SCOPE_FOR VARIABLES)
if(DTWC_BUILD_PYTHON AND NOT DTWC_HIGHS_GPU)
set(BUILD_SHARED_LIBS OFF)
set(BUILD_SHARED_EXTRAS_LIB OFF)
endif()

CPMAddPackage(
NAME highs
URL "https://github.com/ERGO-Code/HiGHS/archive/refs/tags/v1.15.1.tar.gz"
# SHA256 pinned (Task 0.12). Computed 2026-07-08 from the GitHub release
# tarball for the immutable tag v1.15.1 (`curl -sL … | sha256sum`).
URL_HASH SHA256=a840d269dff2fafb371dd247df13ad5e026d7ce3b35ad3dc1eedd59bf0c2fb16
SYSTEM
EXCLUDE_FROM_ALL
OPTIONS
"CI OFF"
"ZLIB OFF"
"BUILD_CXX_EXE OFF"
"BUILD_EXAMPLES OFF"
"BUILD_TESTING OFF"
"FAST_BUILD ON"
)
endblock()

# Fail during configuration, rather than shipping another extension with an
# unresolved libhighs dependency, if HiGHS changes how it honours
# BUILD_SHARED_LIBS.
if(DTWC_BUILD_PYTHON AND NOT DTWC_HIGHS_GPU AND TARGET highs)
get_target_property(_dtwc_highs_library_type highs TYPE)
if(NOT _dtwc_highs_library_type STREQUAL "STATIC_LIBRARY")
message(FATAL_ERROR
"Python packages require statically bundled HiGHS; got "
"${_dtwc_highs_library_type}")
endif()
endif()

# Historically HiGHS <=1.14.0 had a debug assertion (ub_consistent) that
# fired on valid warm-start MIP solves (primal-dual integral bookkeeping
# tolerance 1e-12 too tight after a presolve reset). Retained defensively:
Expand Down
21 changes: 21 additions & 0 deletions dtwc/Problem.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,20 @@ class Problem
std::vector<int> centroids_ind; //!< indices of cluster centroids. [0, Np)

// Constructors:
// GCC emits -Wdeprecated-declarations for in-class initializers of the
// deprecated maxIter / N_repetition fields at every constructor definition.
// Canonical construction must stay silent (F22); caller access of those
// fields must still diagnose. Same push/pop as Problem.cpp accessors.
#if defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wdeprecated-declarations"
#elif defined(__GNUC__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wdeprecated-declarations"
#elif defined(_MSC_VER)
# pragma warning(push)
# pragma warning(disable : 4996)
#endif
Problem() { rebind_dtw_fn(); }
Problem(std::string_view problem_name) : name_{ problem_name }
{
Expand All @@ -287,6 +301,13 @@ class Problem
adopt_loaded_data(loader.load_stored());
refresh_distance_matrix(); // also calls rebind_dtw_fn()
}
#if defined(__clang__)
# pragma clang diagnostic pop
#elif defined(__GNUC__)
# pragma GCC diagnostic pop
#elif defined(_MSC_VER)
# pragma warning(pop)
#endif
Problem(const Problem &) = delete;
Problem &operator=(const Problem &) = delete;
Problem(Problem &&);
Expand Down
6 changes: 5 additions & 1 deletion dtwc/core/llfio_include.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,13 @@

// quickcpplib/ringbuffer_log.hpp installs an unbalanced Clang ignore for
// -Wdeprecated-declarations on Windows. Preserve the caller's diagnostic state
// across the complete third-party include.
// across the complete third-party include. Apple libc++ also deprecates
// std::char_traits<std::byte> inside LLFIO headers; ignore that only while
// the dependency is parsed so a later -Werror=deprecated-declarations still
// diagnoses DTWC++ [[deprecated]] names (F22 / F45).
#if defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wdeprecated-declarations"
#endif

#include <llfio/v2.0/llfio.hpp>
Expand Down
11 changes: 10 additions & 1 deletion scripts/test_f22_cpp_deprecations.py
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,8 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]:
if not arguments:
raise RuntimeError("empty compile command")
compiler = arguments[0].strip('"')
compiler_name = Path(compiler).name.lower()
msvc_slash_flags = compiler_name in ("cl", "cl.exe") or "clang-cl" in compiler_name
context: list[str] = []
index = 1
paired_options = {
Expand All @@ -388,7 +390,14 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]:
"--sysroot",
"-target",
"--target",
# Apple Clang rejects a bare -fopenmp; CMake records
# -Xpreprocessor -fopenmp (and sometimes -Xclang -fopenmp).
"-Xpreprocessor",
"-Xclang",
}
define_include_prefixes = ("-I", "-D", "-U")
if msvc_slash_flags:
define_include_prefixes += ("/I", "/D", "/U")
while index < len(arguments):
argument = arguments[index]
if argument in paired_options:
Expand All @@ -398,7 +407,7 @@ def compile_context(arguments: list[str]) -> tuple[str, list[str]]:
index += 2
continue
if (
argument.startswith(("-I", "/I", "-D", "/D", "-U", "/U"))
argument.startswith(define_include_prefixes)
and '\\"' not in argument
):
context.append(argument.strip('"'))
Expand Down
2 changes: 1 addition & 1 deletion tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ if(TARGET unit_test_deterministic_series)
TEST unit_test_deterministic_series
PROPERTY SKIP_RETURN_CODE)
set(_f15_focused_regex
"F15_TEST_SUPPORT profile=(relaxed|precise|libstdcxx) scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0(.|[\r\n])*All tests passed \\((3[5-9]|[4-9][0-9]|[1-9][0-9][0-9]+) assertions in ([5-9]|[1-9][0-9]+) test cases\\)")
"F15_TEST_SUPPORT profile=(relaxed|precise|libstdcxx|libcxx) scalar=ran row_seeded=ran continuous=ran dense=ran source_audit=ran skips=0(.|[\r\n])*All tests passed \\((3[5-9]|[4-9][0-9]|[1-9][0-9][0-9]+) assertions in ([5-9]|[1-9][0-9]+) test cases\\)")
set_tests_properties(
unit_test_deterministic_series
PROPERTIES
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/test_distance_matrix_csv_contract.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ function(run_cli_route route output_dir threshold expect_mmap)
"${route} distance-matrix output")
require_occurrences(
"${normalized_stdout}"
"\"${output_dir}//conformance_distance_matrix.csv\"" 1
"\"${matrix_path}\"" 1
"${route} exact distance-matrix path")
foreach(execution_marker IN ITEMS
"DTWC++ Clustering"
Expand All @@ -185,7 +185,7 @@ function(run_cli_route route output_dir threshold expect_mmap)
"${route} mmap route")
require_occurrences(
"${normalized_stdout}"
"\"${output_dir}//conformance_distmat.cache\"" 1
"\"${cache_path}\"" 1
"${route} exact mmap cache")
if(NOT EXISTS "${cache_path}")
message(FATAL_ERROR
Expand Down
Loading