[Feature] Support aarch64/arm64 architecture

[SteNicholas]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Paimon C++ currently only builds and is validated on x86_64. aarch64/arm64 has become a mainstream server and desktop platform (Apple Silicon, AWS Graviton, Alibaba Cloud Yitian, Ampere, etc.), and users deploying Paimon readers/writers on these platforms cannot use the library today.

Concretely, the following gaps block first-class aarch64/arm64 support:

1. No architecture detection in the build system. cmake_modules/SetupCxxFlags.cmake consumes PAIMON_CPU_FLAG (with x86, ppc, and armv8 branches), but nothing in the repository ever sets it from CMAKE_SYSTEM_PROCESSOR, so the armv8 tuning path (PAIMON_ARMV8_ARCH_FLAG, CRC feature detection) is effectively dead code and the x86 SIMD defines (PAIMON_HAVE_SSE4_2/AVX2/AVX512) are not guarded by an actual architecture check.
2. Third-party dependencies with x86_64-only artifacts. In cmake_modules/ThirdpartyToolchain.cmake, the lumina dependency hardcodes artifacts/cpp/linux-x86_64/install-root/... paths, so the build cannot link it on aarch64. (JindoSDK already resolves aarch64 artifacts for both Linux and macOS, so it is not a blocker.) The tantivy-based global index is built from Rust via Corrosion and needs verification that the toolchain targets aarch64 correctly.
3. No aarch64 CI coverage. All workflows under .github/workflows/ run on x86_64 GitHub-hosted runners (ubuntu-22.04/ubuntu-24.04), so architecture-specific regressions are invisible. There is at least one known class of portability issues already noted in the codebase: char is unsigned by default on aarch64 (see the workaround comment in cmake_modules/BuildUtils.cmake), which can silently change behavior of code assuming signed char.
4. No arm64 release artifacts. The packaging/release tooling only produces x86_64 outputs, and the documentation does not state supported platforms.

Solution

Proposed incremental plan:

1. Derive PAIMON_CPU_FLAG from CMAKE_SYSTEM_PROCESSOR (x86 / armv8 / ppc) in the CMake setup, so the existing armv8 branch in SetupCxxFlags.cmake becomes reachable and x86 SIMD flags are only applied on x86.
2. Audit dependencies in ThirdpartyToolchain.cmake: select aarch64 artifacts where available (as already done for JindoSDK), build from source or make the dependency optional where no aarch64 artifact exists (e.g. lumina), and verify the Corrosion/Rust tantivy build on aarch64.
3. Add an aarch64 CI job using GitHub's arm64 hosted runners (ubuntu-24.04-arm) that runs the standard build and unit test suite, and fix any test failures uncovered (char signedness, alignment, intrinsics).
4. Extend the release/packaging scripts to produce linux-aarch64 (and macos-arm64) artifacts, and document the supported platform matrix in docs/source/building.rst.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!