Add conservative projection functionality to C-Fortran API - #349
Conversation
- runs but results are incorrect - followed the python example - the petsc mpi comm changes are temp fix for some error
48e54d1 to
113d36d
Compare
replace pcms::Field with pcms::Function
|
I will launch self-hosted build in a few minutes. Currently updating the dependencies [redev]. |
There was a problem hiding this comment.
Pull request overview
This PR extends the C/Fortran API surface to support a cached, mesh-intersection-based conservative projection workflow, and adjusts PETSc object creation/abort communicators to use PETSC_COMM_SELF for sequential PETSc objects (improving correctness for MPI runs and GPU/Kokkos paths).
Changes:
- Switch sequential PETSc Vec/Mat/KSP creation and
CHKERRABORTcalls fromPETSC_COMM_WORLDtoPETSC_COMM_SELFin conservative transfer components. - Add a new conservative projection handle + functions to the C API, and expose them through the SWIG-generated Fortran module/wrappers.
- Implement conservative projection context creation, apply, and destroy logic in the C API using Omega_h + PCMS MeshFields + PETSc.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/pcms/transfer/omega_h_mc_rhs_integrator.cpp | Use PETSC_COMM_SELF for sequential PETSc RHS vector creation/assembly aborts. |
| src/pcms/transfer/omega_h_mass_integrator.cpp | Use PETSC_COMM_SELF for sequential PETSc mass matrix creation/assembly aborts. |
| src/pcms/transfer/omega_h_intersection_rhs_integrator.cpp | Use PETSC_COMM_SELF for sequential PETSc RHS vector creation/assembly aborts. |
| src/pcms/transfer/conservative_projection_solver.cpp | Use PETSC_COMM_SELF for sequential KSP creation/solve error handling. |
| src/pcms/fortranapi/pcms_interpolator.f90 | Expose new conservative projection handle and entry points to Fortran (SWIG output). |
| src/pcms/fortranapi/interpolator.i | Add SWIG interface declarations for the conservative projection C API. |
| src/pcms/fortranapi/interpolator_wrap.c | Add SWIG-generated C wrapper glue for the new conservative projection API. |
| src/pcms/capi/interpolator.h | Declare/document the new conservative projection C API and handle type. |
| src/pcms/capi/interpolator.cpp | Implement the new conservative projection C API (context + create/apply/destroy). |
Suppressed comments (1)
src/pcms/capi/interpolator.cpp:223
- pcms_conservative_projection_get_target_size() unconditionally casts and dereferences projection.pointer. If the handle is null/invalid this will crash; return 0 and emit an error instead.
int pcms_conservative_projection_get_target_size(
PcmsConservativeProjectionHandle projection)
{
auto* ctx =
reinterpret_cast<ConservativeProjectionContext*>(projection.pointer);
return ctx->target_space->GetLayout()->GetNumOwnedDofHolder();
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| int pcms_conservative_projection_get_source_size( | ||
| PcmsConservativeProjectionHandle projection) | ||
| { | ||
| auto* ctx = | ||
| reinterpret_cast<ConservativeProjectionContext*>(projection.pointer); | ||
| return ctx->source_space->GetLayout()->GetNumOwnedDofHolder(); | ||
| } |
| PcmsConservativeProjectionHandle pcms_create_conservative_projection( | ||
| const char* source_mesh_name, int source_order, const char* target_mesh_name, | ||
| int target_order) | ||
| { | ||
| auto* ctx = new ConservativeProjectionContext( | ||
| source_mesh_name, target_mesh_name, source_order, target_order); | ||
| return {reinterpret_cast<void*>(ctx)}; | ||
| } |
| * @brief Apply conservative projection | ||
| * @param projection Handle to the projection | ||
| * @param source_data Flat array of source field values (size = source_size) | ||
| * @param source_size Number of source DOF holders | ||
| * @param target_data Flat array to receive target field values (size = | ||
| * target_size) | ||
| * @param target_size Number of target DOF holders |
| void pcms_conservative_projection_apply( | ||
| PcmsConservativeProjectionHandle projection, void* source_data, | ||
| int source_size, void* target_data, int target_size) | ||
| { | ||
| auto* ctx = | ||
| reinterpret_cast<ConservativeProjectionContext*>(projection.pointer); | ||
|
|
||
| // Copy source data into internal field | ||
| auto source_view = pcms::Rank2View<const pcms::Real, pcms::HostMemorySpace>( | ||
| reinterpret_cast<pcms::Real*>(source_data), source_size, 1); | ||
| ctx->source_field.SetDOFHolderDataHost(source_view); | ||
|
|
||
| // Apply conservative projection | ||
| ctx->projection->Apply(ctx->source_field, ctx->target_field); | ||
|
|
||
| // Copy result back to user buffer | ||
| auto target_view = ctx->target_field.GetDOFHolderDataHost(); | ||
| auto flat = pcms::FlattenToRank1View(target_view); | ||
| std::memcpy(target_data, flat.data_handle(), | ||
| static_cast<std::size_t>(target_size) * sizeof(double)); | ||
| } |
jacobmerson
left a comment
There was a problem hiding this comment.
I commented in a number of places, but the biggest thing is that with the new API, you should have everything except the creation function be called on a generic PcmsTransferOperatorHandle which holds the base pointer to the TransferOperator. That way the same functions can be used for every field transfer method in the exact same way. I.e., the only thing you need to do to swap out the transfer method you are using is swap out the constructor.
|
|
||
| } // namespace | ||
|
|
||
| PcmsConservativeProjectionHandle pcms_create_conservative_projection( |
There was a problem hiding this comment.
Can this return a TransferOperatorHandle? With the new API, all of the transfer operators should be able to be used the same way after construction.
| return {reinterpret_cast<void*>(ctx)}; | ||
| } | ||
|
|
||
| int pcms_conservative_projection_get_source_size( |
There was a problem hiding this comment.
pcms_transfer_get_source_size(TransferOperatorHandle)?
| return ctx->source_space->GetLayout()->GetNumOwnedDofHolder(); | ||
| } | ||
|
|
||
| int pcms_conservative_projection_get_target_size( |
There was a problem hiding this comment.
pcms_transfer_get_target_size(TransferOperatorHandle)?
| return ctx->target_space->GetLayout()->GetNumOwnedDofHolder(); | ||
| } | ||
|
|
||
| void pcms_conservative_projection_apply( |
There was a problem hiding this comment.
pcms_transfer_apply(TransferOperatorHandle)
|
|
||
| // Copy result back to user buffer | ||
| auto target_view = ctx->target_field.GetDOFHolderDataHost(); | ||
| auto flat = pcms::FlattenToRank1View(target_view); |
There was a problem hiding this comment.
I suspect we can probably avoid this memcopy if the target field is constructed based on the target field pointer. I think the XGC field data is non-owning specifically for this reason, however I implemented that a while ago so it needs to be checked.
| static_cast<std::size_t>(target_size) * sizeof(double)); | ||
| } | ||
|
|
||
| void pcms_destroy_conservative_projection( |
There was a problem hiding this comment.
pcms_destroy_transfer_operator(...
den_neutandtemp_neutfromxgc.neutral.*.bpfiles.developbranch