Update directory structure and rename from Cppyy->CppJIT by aaronj0 · Pull Request #14 · compiler-research/cppjit

aaronj0 · 2026-08-06T16:37:20Z

No description provided.

aaronj0 · 2026-08-06T16:37:58Z

As discussed/planned, following this merge we can squash the history and begin active development.

vgvassilev · 2026-08-06T17:21:13Z



-namespace CPyCppyy {
+namespace PyCppJIT {


Is that the middle layer? I am wondering if we have better options for a namespace name to signal that.

aaronj0 · 2026-08-06T22:36:13Z

@vgvassilev I'm continuing our discussions here, so I can propose the design we seem to be converging upon:

cppjit namespace design: cppjit::{cpy_rt, interop}

We use a single C++ namespace tree,
cppjit::, with two members today: cppjit::cpy_rt (spoken as "cpy
runtime") is the replacement for CPyCppyy and holds the CPython runtime
layer, and cppjit::interop is reserved for what is currently
clingwrapper and all interactions with CppInterOp. clingwrapper is renamed
to the interop wrapper (interop_wrapper.cxx)

Namespace	What it is	Sources
`cppjit::cpy_rt`	the CPython runtime: proxies, converters, executors, pythonizations — everything that includes `Python.h`	`src/cpy_rt/`
`cppjit::interop`	reflection and JIT over CppInterOp — what the interop wrapper implements	`src/backend/` (goes away post-squash)

What one writes and reads as a cppjit developer

The two layers are siblings under cppjit::, so code in cpy_rt reaches
the reflection API as plain interop:: without full qualification:

// src/cpy_rt/Converters.cxx
namespace cppjit::cpy_rt {

bool VectorConverter::SetArg(PyObject* pyobject, Parameter& para, CallContext* ctxt)
{
    // sibling namespace, no full qualification needed
    interop::TCppScope_t scope = interop::GetScope("std::vector<double>");
    ...
}

} // namespace cppjit::cpy_rt

Someone extending cppjit from outside includes the installed headers
(they keep the layer prefix, cppjit_backend/include/cpy_rt/) and writes:

#include "cpy_rt/API.h"

class MyConverter : public cppjit::cpy_rt::Converter {
    bool SetArg(PyObject*, cppjit::cpy_rt::Parameter&,
                cppjit::cpy_rt::CallContext* = nullptr) override;
};
cppjit::cpy_rt::RegisterConverter("MyType", ...);

From Python and from JIT-side C++ the same tree shows up as nested
namespace proxies:

import cppjit

cppjit.cppdef("int f(int x) { return x + 1; }")
cppjit.gbl.f(41)

# the reflection API is a normal namespace under gbl
scope = cppjit.gbl.cppjit.interop.GetScope("std::vector<int>")

The naming conventions in one place:

Surface	Convention	Example
C++ namespaces	`cppjit::<layer>`	`cppjit::cpy_rt`, `cppjit::interop`
Layer macros/guards	`CPY_RT_`, `CPPJIT_`	`CPY_RT_API_H`, `CPPJIT_IMPORT`
Python package	`cppjit` (+ top-level `cppjit_backend`, `libcppjit`)	`import cppjit`
tp_names / reprs	`cppjit.<Type>`	`cppjit.CPPInstance`
Module strings	`cppjit.gbl`, `cppjit.gbl.std`	pickling, `__module__`
Hook protocol	`__cppjit_*`	`__cppjit_pythonize__`
Env vars	`CPPJIT_*`	`CPPJIT_API_PATH`

How python-fortran interop would fit

The split between the two namespaces is also the extension seam.
cppjit::interop has no Python.h dependency (that separation is what
lets the merged library exist in the first place), and cppjit::cpy_rt
does not care what source language an entity came from — converters and
executors are keyed on reflected types. So adding fortran support is
additive on both sides:

On the compiler side, interop gains a Fortran provider (Flang-based),
either as new entry points in cppjit::interop or as a sibling
implementing the same reflection contract. The existing C++ paths and
CppInterOp stay as they are.
On the runtime side, fortran-backed entities surface to Python through
the cpy_rt machinery that already exists: new converter and executor
registrations via the same public API third parties use, not changes
to existing ones. If fortran needs runtime state of its own it gets a
sibling namespace and directory:

src/
├── cpy_rt/      cppjit::cpy_rt      (CPython runtime)
├── interop/     cppjit::interop     (reflection/JIT, post-squash name)
└── fortran_rt/  cppjit::fortran_rt  (hypothetical fortran runtime — additive)

On the Python side a cppjit.fortran submodule exposes the fortran
namespace the way gbl exposes C++, and the package absorbs it
without renames.

The invariant that makes this "unintrusive": new languages add siblings
(namespaces, directories, registrations) and don't rename or restructure
existing members. That is what motivated the flat src/ layout plus the
cppjit:: umbrella, and no api//include/ tree is created until we
converge on an agreed-upon public API surface.

cc @guitargeek

guitargeek · 2026-08-07T08:40:18Z

Sounds good! cppjit::runtime would read less clunky to me compared to cppjit::cpy_rt, and it's obvious that this is a CPython extension anyway, since there is no pypy version anymore.

Anyway I have no strong preference at all because there is no public API, but if there would be one at some point, I'd definitely prefer #include "cppjit/API.h" over #include "cpy_rt/API.h". Or even cppjit.h. I mean the name API and the fact that it's a public header file is also kind of redundant.

vgvassilev · 2026-08-07T08:52:42Z

# the reflection API is a normal namespace under gbl
scope = cppjit.gbl.cppjit.interop.GetScope("std::vector<int>")

This is a bit clunky but I guess we can solve that later not now...

vgvassilev · 2026-08-07T08:54:40Z

Sounds good! cppjit::runtime would read less clunky to me compared to cppjit::cpy_rt, and it's obvious that this is a CPython extension anyway, since there is no pypy version anymore.

Anyway I have no strong preference at all because there is no public API, but if there would be one at some point, I'd definitely prefer #include "cppjit/API.h" over #include "cpy_rt/API.h". Or even cppjit.h. I mean the name API and the fact that it's a public header file is also kind of redundant.

Structurally, interop and cpy_rt are both performed at runtime. So having a generic name runtime can be confusing if we want to distinguish where the api lives.

guitargeek · 2026-08-07T08:55:49Z

# the reflection API is a normal namespace under gbl
scope = cppjit.gbl.cppjit.interop.GetScope("std::vector<int>")
This is a bit clunky but I guess we can solve that later not now...

Not an actual problem for now fortunately, because we won't expose the public C++ API

aaronj0 · 2026-08-07T09:00:06Z

Structurally, interop and cpy_rt are both performed at runtime. So having a generic name runtime can be confusing if we want to distinguish where the api lives.

sI would prefer cpyruntime over cpy_rt as that is more readable for the same syllables

vgvassilev · 2026-08-07T09:10:34Z

Structurally, interop and cpy_rt are both performed at runtime. So having a generic name runtime can be confusing if we want to distinguish where the api lives.

sI would prefer cpyruntime over cpy_rt as that is more readable for the same syllables

If we are discussing folder name then that's fine, for a namespace I think it is a bit long. In that sense (assuming we follow the 80 col llvm rule) people will probably do using namespace cpyruntime which will hide again which belongs where when reading it.

EDIT: cpyrt is fine for me.

aaronj0 · 2026-08-07T09:16:45Z

EDIT: cpyrt is fine for me.

Sounds good. @guitargeek?

guitargeek · 2026-08-07T09:18:13Z

Sure!

github-actions · 2026-08-07T09:29:19Z

Test Results

Configuration	Result
macos-26-intel-llvm21-py3.14-cxx20	`no summary (build or setup failed)`
macos-26-llvm21-py3.14-cxx20	`no summary (build or setup failed)`
ubuntu-24.04-llvm20-py3.14-cxx20-cling	`no summary (build or setup failed)`
ubuntu-24.04-llvm21-py3.14-cxx20-vg	`no summary (build or setup failed)`
ubuntu-24.04-llvm22-py3.14-cxx20	`no summary (build or setup failed)`

vgvassilev · 2026-08-07T09:38:59Z

Test Results

Configuration Result
macos-26-intel-llvm21-py3.14-cxx20 no summary (build or setup failed)
macos-26-llvm21-py3.14-cxx20 no summary (build or setup failed)
ubuntu-24.04-llvm20-py3.14-cxx20-cling no summary (build or setup failed)
ubuntu-24.04-llvm21-py3.14-cxx20-vg no summary (build or setup failed)
ubuntu-24.04-llvm22-py3.14-cxx20 no summary (build or setup failed)

For maybe a future improvement, we should have this either to the bottom of the PR description which has the last run and in <details> tag all the previous runs ordered; or as the first comment including the relevant commit hash it was built against. This way for multirun setups we won't have to scroll up and down and click on "load" to see the full integration history of a given change.

aaronj0 · 2026-08-07T09:43:24Z

For maybe a future improvement, we should have this either to the bottom of the PR description which has the last run and in <details> tag all the previous runs ordered; or as the first comment including the relevant commit hash it was built against. This way for multirun setups we won't have to scroll up and down and click on "load" to see the full integration history of a given change.

This is currently posts the last run results, ensuring it is always at the end of the PR (so it follows a force push for e.g), and the intermediate runs are discarded: at any given point in time there is only one "Test Results" displayed by github actions. What do you mean by multirun setups

aaronj0 · 2026-08-07T10:44:50Z

Pending work-items for this PR:

improve test directory structure
drop dead files
clang-format after approval
switch to using namespaces in source files

This should in the end give us a relatively "minimal" starting point for development after squashing the history. Some post squash items have been recorded in #15

[build] Flatten the CPyCppyy include and src trees into src/CPyCppyy

a2cec93

aaronj0 requested review from guitargeek and vgvassilev August 6, 2026 16:37

vgvassilev reviewed Aug 6, 2026

View reviewed changes

Rename the cppyy namespaces and packages to cppjit

9702f93

aaronj0 force-pushed the pre-squash-layout branch from fe26006 to 9702f93 Compare August 7, 2026 09:25

Uh oh!

Conversation

aaronj0 commented Aug 6, 2026

Uh oh!

aaronj0 commented Aug 6, 2026

Uh oh!

vgvassilev Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

aaronj0 commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

cppjit namespace design: cppjit::{cpy_rt, interop}

What one writes and reads as a cppjit developer

How python-fortran interop would fit

Uh oh!

guitargeek commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vgvassilev commented Aug 7, 2026

Uh oh!

vgvassilev commented Aug 7, 2026

Uh oh!

guitargeek commented Aug 7, 2026

Uh oh!

aaronj0 commented Aug 7, 2026

Uh oh!

vgvassilev commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaronj0 commented Aug 7, 2026

Uh oh!

guitargeek commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Test Results

Uh oh!

vgvassilev commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

aaronj0 commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaronj0 commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

aaronj0 commented Aug 6, 2026 •

edited

Loading

guitargeek commented Aug 7, 2026 •

edited

Loading

vgvassilev commented Aug 7, 2026 •

edited

Loading

vgvassilev commented Aug 7, 2026 •

edited

Loading

aaronj0 commented Aug 7, 2026 •

edited

Loading

aaronj0 commented Aug 7, 2026 •

edited

Loading